Decoder. plain-English AI glossary

Reference answer

● Core

The correct answer to a question, written by a human, that you use to grade automated responses.

Think of it like

The answer key to a test — you check whether students got it right by comparing to this.

Example

For "What is the capital of France?" the reference answer is "Paris." If your model says "Paris" or "The capital of France is Paris," both grade as correct against this reference.

How it actually works

Reference answers are expensive — humans have to write them. Some systems use multiple references (Paris, Île-de-France) to allow variation. The danger: a good answer that differs from the reference gets marked wrong, so your eval underestimates model quality. This is why reference-free evaluation exists.

For product teams

You need reference answers to grade at scale, but they're a bottleneck. Budget time to create them.

For engineers

Store as gold annotations. Use fuzzy matching or embedding similarity to accept minor paraphrases.

Used in evals with baselines.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome