Data contamination
When test questions leak into training data, so a model’s scores look better than its real ability.
Think of it like
Like a student who secretly saw the exam beforehand — the perfect grade tells you nothing about what they actually learned.
Example
A benchmark’s questions turn up in the web-scraped training set, and the model “aces” them by recall, not reasoning.
How it actually works
Because pretraining scrapes huge swaths of the web, public benchmarks often end up inside the training data, inflating evaluation scores. It’s a major reason leaderboard numbers can mislead and why fresh, held-out, or private test sets matter. Detecting it is hard — you’re looking for overlap in trillions of tokens — so teams increasingly favor recently-created or canary-marked benchmarks.
For product teams
Why you shouldn’t trust a shiny benchmark number at face value — the test may have leaked into training.
For engineers
Overlap between eval and training data inflating metrics; mitigated by held-out/fresh/canary test sets and dedup.
Related
- Deduplication — Fought with deduplication.
- Evals — Undermines evals.
- Pretraining — A hazard in pretraining.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome