Probability distribution
A spread of chances across all the options — how the model splits its confidence over every possible next token.
Think of it like
A weighted die: not every face equally likely, and the weights tell you where the model’s leaning.
Example
After “The sky is”, the model might put 60% on “blue”, 10% on “clear”, and thin slices on thousands of others.
How it actually works
At each step the model outputs a number for every token in its vocabulary, then softmax turns those into probabilities that sum to one. Sampling picks from this distribution. Temperature and top-p are just knobs that reshape it before the pick.
For product teams
The model rarely “knows” one answer — it holds a spread, and your settings decide how boldly it commits.
For engineers
The softmax output over the vocabulary; the categorical distribution sampled at each decode step.
Related
- Softmax — Produced by softmax.
- Temperature — Sharpened or flattened by temperature.
- Sampling — Picked from via sampling.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome