Log probability
Also called Log-prob
The model’s confidence in a token, written on a stretched-out scale so tiny odds are easy to handle.
Think of it like
Like measuring earthquakes on a log scale — it lets you compare a whisper and a shout without the numbers getting unwieldy.
Example
A developer inspects the logprobs and sees the model was 90% sure of “Paris” but split three ways on the next word — a hint it’s unsure.
How it actually works
Probabilities multiply and quickly underflow to near-zero, so we work in log space where multiplying becomes adding. Summing token logprobs gives a sequence score; averaging and negating gives the loss that training minimizes and that perplexity is built from.
For product teams
Exposing logprobs lets you flag low-confidence answers instead of shipping every guess as gospel.
For engineers
log(p) of a chosen token under the model’s distribution; summed over a sequence for scoring, negated and averaged for cross-entropy loss.
Related
- Perplexity — Averaged and exponentiated, these give perplexity.
- Logits — The raw scores behind them come from logits.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome