Decoder. plain-English AI glossary

Eval Set

● Core

A fixed dataset of examples you use to measure model quality—separate from training data to prevent overfitting.

Think of it like

Like the final exam of a class: fixed questions, given after students learn.

Example

You split your 10,000 labeled examples: 7,000 for training, 3,000 for eval. You train on the 7,000, measure accuracy on the 3,000.

How it actually works

Using a separate eval set is essential for detecting overfitting. If you eval on training data, you can't tell if the model learned generalizable patterns or just memorized. Standard split is 70–80% training, 10–15% validation, 10–15% test. Some datasets call the last split 'test' to emphasize it's held out.

For product teams

Eval sets tell you if your model works on unseen data. Without them, you have no confidence.

For engineers

Never touch eval sets during training; don't use them for hyperparameter tuning (use validation sets instead).

Gold Set — Often the same thing.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome