Benchmark contamination, and five other ways an AI score lies: what we found in our own
Benchmark contamination is one of six ways an AI score misleads. Each failure, a real example from our own benchmark, and the check that would have caught it.
By overads Inc., the team building gloofy
Benchmark contamination is when a model has seen the test, or something close to it, before it is scored, so its result measures memory rather than skill. It is the best-known way an AI score misleads, but not the only one, and often not the most likely. Answer keys can be wrong, the test harness can leak the answer, the comparison can be unfair. We have made most of these mistakes in our own benchmark and published each one, and every one has a cheap check that would have caught it sooner.
Why benchmark contamination is back in the news
In an interview with Quanta Magazine on 16 October, Moritz Hardt, a director at the Max Planck Institute for Intelligent Systems, argued that the benchmarking ecosystem is broken. His point about contamination is straightforward: machine learning stopped keeping test sets locked away, test sets ended up online, and model builders now tune training data towards known benchmarks without disclosing it. One sign he points to is new models scoring well on older benchmarks and worse on ones released shortly after.
Hardt makes a second point that matters as much. Absolute scores are fragile; a small change to a test set can move accuracy a lot. Relative rankings have historically been more stable across datasets, which is why benchmarks still worked. He also notes that this stability held when models were trained on similar data, which is no longer reliably true.
Three days later, Contrary Research revisited a leaderboard episode. In April 2025, an experimental version of Meta's Llama 4 Maverick ranked second on the Arena leaderboard; the publicly released model, retested, ranked 32nd. The research piece also reports that more than 225 million dollars traded on prediction markets that resolve on that leaderboard. Scores have consequences, so the ways they mislead are worth knowing.
Six ways an AI benchmark score misleads
| Failure | What it looks like | Our own example | The check |
|---|---|---|---|
| Contamination | The model saw test items, or near copies, in training | Our exam has zero training overlap and was published before the model | Hash every test item against the training set before training |
| Answer leakage in the harness | The evaluator hands the model the answer | We reported 0.947 on a task; the harness was sending the correct answer with the brief. Measured properly: 0.04 | Read the exact prompts the model received, not just the scores |
| Wrong answer key | The model is marked wrong for being right | Our v3.1 key was about 19 percent wrong; Claude Opus overrode 142 of 750 gold fields on review | Have a second annotator re-check every disagreement |
| Prompt familiarity | A model scores higher in the words it was trained on | Our older model scored 0.745 on its own prompt and 0.675 on the shared one | Give every model the same specification and report that score |
| Unfair baseline | A comparison model is set up to fail | We first published the untrained base at 0.000; given the shared specification it scores 0.620 | Run baselines through the same harness with the same instructions |
| Variant shopping | A lab scores many private versions and ships a different one | The Maverick case above (not ours) | Score the exact public file people download |
Our own rows are not hypothetical. Each is recorded in our build log and the benchmark's history, with the corrected number next to the original.
What a wrong answer key does to a leaderboard
The answer key problem is the least discussed and, in our experience, the most damaging. Our first exam of 150 real published ads was keyed by three blind annotators. A later review by a higher-consistency annotator, Claude Opus, which agreed with itself 96.7 percent of the time across three passes, overrode 142 of 750 gold fields: about 19 percent. Re-scoring Mistral Large 3 on the corrected key lifted it by 0.052. The errors were in the key, not the models.
It also changed our own position. On the old key, our model looked tied for first. On the corrected exam, v3.3, our released 14B adapter scores 0.816 against 0.825 for Mistral Large 3, second of eight and tied within measurement error. That is Hardt's point in miniature: the absolute numbers moved a lot, and the order near the top moved too. We published the correction rather than keeping the better-looking table.
An earlier, smaller version of the same problem: when our labelling rules improved and the answer key did not follow, one model scored 0.46 on the old key and 0.64 on the corrected one, without a single weight changing. We now ship both key versions with the rules that produced each.
Contamination you cannot see: who wrote the key
There is a subtler form of contamination: the answer key and a contestant sharing a source. Our original annotators were Claude agents, so Claude's entry on that exam was graded against a key its own model family wrote, an advantage no other entry had and the likeliest reason it scored above the annotator agreement ceiling. The corrected key was re-adjudicated by Claude Opus, so the same caveat applies to any Claude model measured on it. We cover this self-preference problem, and the judging protocol we use against it, in using an LLM as a judge.
Where gloofy fits, and where it does not
gloofy is a set of small open models, listed with their licences under the gloofy models, and marketing-bench is the public exam we measure them on. The benchmark exists because almost every marketing AI product scores itself on an evaluation nobody outside the company can see. Ours ships with the harness, the data and the commands to reproduce every number, including the ones that went against us.
It is not immune to anything in the table above. We have made four of those six mistakes ourselves and caught them only by rerunning and reading our own outputs. Closed frontier models have not been measured on the corrected exam yet, so the current leaderboard says nothing about how gloofy compares with them. More on how we measure collects in the benchmarks and evaluation hub.
How to check any AI benchmark claim this week
- Ask whether the test was public before the model was trained. If it was, assume some contamination unless the vendor shows an overlap check.
- Ask who wrote the answer key. If it is the vendor, or a model from the same family as the winner, discount the win.
- Ask for the prompts. A score without the exact instructions given to each model is not comparable.
- Look for corrections. A benchmark that has never published one has probably not looked.
- Prefer rankings to absolute scores, and treat close rankings on a small test as ties.
FAQ
What is benchmark contamination?
Benchmark contamination is when a model has seen test items, or close copies of them, during training, so its score reflects memorisation rather than ability. It happens easily when test sets are published online and training data is not disclosed. The usual check is to compare test items against the training data, for example by hashing their content.
How can you tell if an AI benchmark result is contaminated?
You often cannot from outside. Warning signs include a model scoring much better on an older public benchmark than on a newer one, a test published before the model was trained with no overlap check, and a vendor that will not share its prompts or data. A benchmark with a reproducible harness lets anyone rerun the result.
Are AI benchmark scores useless?
No, but absolute scores are fragile. Researcher Moritz Hardt notes that relative rankings between models have historically been more stable across test sets than the scores themselves. Treat a single number with caution, treat close rankings on a small test as ties, and trust benchmarks that publish corrections.
What is the most common benchmark mistake?
In our experience, an incorrect answer key, more than contamination. Our own 150-ad exam turned out to be about 19 percent mislabelled when re-checked, which changed scores and rankings. Harness bugs that leak the answer to the model are a close second. Both are found only by reading the raw prompts and outputs.