Perplexity
A score for how surprised a model is by text — lower means it saw the words coming.
Think of it like
Like a reader finishing your sentences: if they nail every next word, they’re barely surprised; if they keep guessing wrong, they’re baffled.
Example
A team compares two language models on the same book; the one with perplexity 12 predicts the text far more comfortably than the one at 45.
How it actually works
Perplexity is basically the exponent of the average per-token loss — roughly, the number of equally-likely choices the model felt it was picking between at each step. It’s handy for comparing models on the same data, but it rewards fluency, not truth: a model can be smoothly confident and still wrong.
For product teams
A quick, cheap health check on a model or dataset, but never mistake low perplexity for “correct.”
For engineers
Exp of the mean negative log-likelihood per token; only comparable across models with the same tokenizer and test set.
Related
- Log probability — It’s computed from the model’s log probabilities.
- Hallucination — A fluent-but-wrong model can still score well.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome