Benchfolio

Method v1 · Sep 23, 2026

How scores work

Every number on Benchfolio comes from public benchmark results, combined with the rules below. The code that applies them is open source, and every daily update is a reviewed pull request.

Rules

  1. We track a curated set of frontier models. The list is open, in data/models.ts.
  2. For each benchmark, a model’s result is its best reported variant, such as its highest reasoning setting. The variant is shown next to every result.
  3. Each benchmark is standardized across tracked models: how far a model sits above or below the average, in standard deviations (a z-score). A hard test and an easy test then count equally, and human-vote ratings fit the same scale. Unbounded results, like METR’s time horizons, are compared on a log scale.
  4. A benchmark counts only when at least 5 tracked models have a result on it.
  5. A category score is the average z-score across that category’s benchmarks. A model needs results on at least half of them, and at least two when a category has two or more. Otherwise it shows as n/a.
  6. Scores run from 0 to 100. 100 is today’s category leader and 0 is the lowest tracked model, so scores are relative to the current frontier and move when new models arrive. Equal scores share a rank.
  7. Overall is the average of the six category scores and needs at least 4 of them.

Benchmarks by category

CategoryBenchmarkSource
CodingDeepSWEEpoch AI
CodingTerminal-BenchEpoch AI
CodingFrontierCodeEpoch AI
CodingSWE-bench VerifiedEpoch AI
CodingLMArena CodingLMArena
AgentsAPEX-AgentsEpoch AI
AgentsOSWorld 2.0Epoch AI
AgentsRemote Labor IndexEpoch AI
AgentsMETR Time HorizonsEpoch AI
ReasoningGPQA DiamondEpoch AI
ReasoningARC-AGI-2Epoch AI
ReasoningSimpleBenchEpoch AI
ReasoningHumanity’s Last ExamEpoch AI
MathFrontierMath Tiers 1–3Epoch AI
MathFrontierMath Tier 4Epoch AI
MathOTIS Mock AIMEEpoch AI
MathProofBenchEpoch AI
MathLMArena MathLMArena
WritingLMArena Creative WritingLMArena
WritingLMArena Instruction FollowingLMArena
DesignLMArena WebDevLMArena

Known limits

  • Averaging over missing results can flatter a model that was only tested where it is strong. The coverage rule reduces this. A statistical model that handles missing results directly (item response theory) is planned for method v2.
  • Writing and Design come only from people voting on LMArena, not from automated tests.

Sources and credits

Some results in Epoch AI’s data come from other projects and keep their original licenses, for example Terminal-Bench under Apache 2.0.

Sponsors

Benchfolio is funded by up to two clearly labeled sponsor placements. Sponsors never appear inside the rankings. Companies whose models we rank cannot sponsor.

Changelog

v1 · Sep 23, 2026 · First public method.