Stochastic
Involving randomness — the reason the same prompt can give you different answers.
Think of it like
Rolling dice rather than reading a ruler: the outcome has a random element baked in.
Example
Ask the same creative question twice and get two different poems — that’s stochastic sampling at work.
How it actually works
Generation is stochastic because the model samples from a probability distribution rather than always taking the top choice. Temperature and top-p control how random. It’s a feature for creativity and a headache for reproducibility — set temperature to zero (or fix a seed) to tame it.
For product teams
It’s why outputs vary run to run — lower the temperature when you need consistency.
For engineers
Sampling-based decoding introduces randomness; contrast with deterministic (greedy) decoding.
Related
- Determinism — The opposite is determinism.
- Sampling — Comes from sampling.
- Temperature — Tuned by temperature.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome