Overfit to Eval
Tuning a model until it aces your test set while quietly getting worse at everything else.
Think of it like
Memorizing last year’s exam paper — you nail that test and learn nothing useful.
Example
A team keeps tweaking prompts to raise a benchmark score, then finds the model got worse on real user questions.
How it actually works
When you make many decisions based on one evaluation set, you slowly fit to its quirks rather than to the real task — the benchmark rises while true quality stalls or drops. It’s subtle because the number keeps going up. Defenses include held-out sets you look at rarely, rotating fresh evals, and judging on live traffic.
For product teams
A rising benchmark can be a mirage; protect a private eval you don’t optimize against so you know what’s real.
For engineers
Metric inflation from repeated selection against a fixed eval set; mitigate with held-out and rotating evaluations.
Related
- Overfitting — A close relative of overfitting.
- Holdout — Why you keep a fresh holdout.
- Train-Test Leakage — Worsened by train-test leakage.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome