Top-p
Also called Nucleus sampling
A sampling limit that keeps only the smallest set of top choices that together cover, say, 90% of the odds — then picks from those.
Think of it like
Only considering the shortlist of front-runners, however long or short that shortlist needs to be.
Example
Top-p 0.9 ignores the long tail of unlikely words but keeps genuine contenders in play.
How it actually works
Unlike top-k’s fixed count, top-p adapts: when the model is confident the shortlist is tiny, when it’s unsure the shortlist grows. It trims the improbable tail that can produce nonsense while still allowing variety among reasonable options. Many teams tune top-p instead of, or alongside, temperature.
For product teams
A safer variety dial than temperature alone — cuts nonsense while keeping natural variation.
For engineers
Nucleus sampling: restrict to the smallest token set with cumulative probability ≥ p, then renormalize.
Related
- Top-k — A gentler cousin of top-k.
- Temperature — Works alongside temperature.
- Sampling — One knob on the sampling step.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome