How to Read AI Model Benchmarks

“The new model beats our current one by a few points in the ranking. Is that reason enough to migrate?” Before answering, check where those points come from. Benchmarks, meaning public sets of test tasks, make sense as a research tool. They lose that sense when a lab score lands on a sales slide as a forecast of how the model will handle a specific company’s invoices, contracts or service tickets.

Three mechanisms account for most of the gap between a ranking and work inside a company.

Contamination: the model knows the answers

Benchmarks are published on the internet, and training data is collected from the internet. The longer a test set circulates publicly, the more likely it is that the model has already seen the tasks or their solutions during training.

In May 2024 Scale AI prepared GSM1k, 1,205 new grade-school math problems matched to the popular GSM8K set in difficulty, number of steps and length.¹ The most overfitted model families lost up to a dozen or so percentage points on the fresh problems, leading models far less. An overfitted model still solves new problems, just worse than the public score promised.

The same happened with SWE-bench Verified (software engineering benchmark), a set of 500 issues from open source repositories that served for two years as the unofficial leaderboard for coding. In February 2026 OpenAI announced it would stop reporting results on that set, because growing contamination and flaws in the tests themselves meant the differences at the top of the table no longer carried information. As an alternative it pointed to SWE-bench Pro, released by Scale AI in September 2025, where part of the tasks is kept in a closed pool. At launch the best models scored below 25% there.²

The benchmark measures a different skill

MMLU (Massive Multitask Language Understanding), for years the most-cited general-knowledge set, consists of multiple-choice questions from 57 subjects. In 2024 a research team manually reviewed a sample of 5,700 questions and published a corrected version called MMLU-Redux.³ It estimated that about 6.5% of items contain errors, from a wrong answer key to ambiguous questions. In virology more than half of the reviewed items turned out to be faulty, and after filtering out bad questions the order of models in the ranking could change.

• 5,700MMLU questions reviewed by hand by the MMLU-Redux team
• c. 6.5%of items with an error: a wrong answer key or an ambiguous question
• over 50%faulty items in the reviewed virology subset

Even with a correct key a heavier difference remains. A task in a company rarely has four answer options. More often it is twenty pages of a poor-quality scan, industry jargon, internal abbreviations, a requirement to cite the source of every number and a consequence that shows up only in the books. Picking the right answer from a list translates only partly into that kind of work.

Variance: the same model, different results

A benchmark score describes the model together with its surroundings: the system prompt, available tools, number of attempts, how answers are verified and the token limit. In agentic tasks this layer accounts for a large share of the difference between published scores, and it is sometimes described in a single footnote sentence.

Most entries on public leaderboards come from the providers themselves and have never been independently reproduced. In April 2025 researchers from Cohere Labs, AI2 and several universities described in “The Leaderboard Illusion” the practice of testing many private model variants before publication and showing only the best one. Before the Llama 4 release Meta tested 27 variants this way.⁴ The leaderboard organizers accepted some of the criticism and disputed some, but result selection remains a problem the reader of a ranking faces alone.

Models also change under a fixed name. A version update or a change of default parameters can shift behavior on edge cases, so a table from last quarter already describes a slightly different model.

Your own test set

The alternative is less impressive and far more effective:

Preparing such a set takes one to three days of work from someone who knows the domain. With 50 cases, differences of a few percentage points mean nothing, but the set reliably separates a model that handles the company’s documents from one that does not.

Four questions before quoting a score

When a ranking table ends up in a meeting anyway, it goes through a short filter:

1. Who reported the score and whether anyone independent reproduced it.
2. When the task set was created and whether it could already be in training data.
3. Whether the set has a closed pool that allows overfitting to be detected.
4. How the test conditions are described: prompt, tools and number of attempts.

A ranking is enough to narrow the candidate list from twenty to three. The decision on which of the three goes into production is made on the company’s own data. We review candidates and plan your own test set as part of an AI Consultation.

Frequently asked questions