Logits
The raw scores a model gives every possible next token before picking one — un-normalized “how likely is each word” numbers.
Think of it like
The tally of votes for each candidate, before you convert them into percentages.
Example
For “The sky is ___”, “blue” gets a high logit, “refrigerator” a very low one.
How it actually works
The final layer outputs one logit per token in the vocabulary. They’re relative scores, not probabilities yet — softmax turns them into a distribution that sums to one. Sampling knobs like temperature and top-p operate on this distribution to decide how adventurous the final pick is.
For product teams
Deep-internals term; matters mainly when you’re tuning how random or safe outputs are.
For engineers
Pre-softmax activations over the vocabulary; scaled by temperature, filtered by top-p/top-k.
Related
- Softmax — Turned into probabilities by softmax.
- Temperature — Reshaped by the temperature dial.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome