Top-p / Top-k
Also called Nucleus sampling
Knobs that control which words the model is allowed to pick from — trimming the unlikely options to keep output focused or creative.
Think of it like
Telling a chef "pick from your best five ingredients" (top-k) or "pick from ingredients that together make up 90% of what's good" (top-p).
Example
Top-k=50 means only the 50 most likely next words are considered. Top-p=0.9 means words are considered until their probabilities add up to 90%.
How it actually works
After the model produces a probability distribution over all possible next tokens, top-k keeps only the k most likely ones, and top-p (nucleus sampling) keeps the smallest set whose cumulative probability exceeds p. Both prevent the model from picking wildly improbable tokens.
For product teams
Output-shaping controls alongside temperature — useful knobs for tuning response quality.
For engineers
Sampling truncation: top-k keeps k most likely tokens; top-p (nucleus) keeps cumulative-probability set.
Related
- Temperature — Works alongside temperature to shape output.
- Inference — All are sampling knobs applied at inference time.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome