Beam search
A decoding trick that keeps several candidate sentences alive at once and picks the best-scoring whole path.
Think of it like
Like planning a road trip by keeping your top few routes open at each junction instead of committing to the first turn that looks good.
Example
An older translation system runs beam width 5, exploring five partial translations in parallel before settling on the most probable finished one.
How it actually works
Greedy decoding commits to the single best token each step and can paint itself into a corner; beam search hedges by tracking the k highest-scoring sequences and expanding all of them. It shines in translation and captioning, but for open-ended chat it tends to produce bland, repetitive text, which is why sampling methods largely replaced it.
For product teams
Great when there’s one “right” output like a translation; wrong when you want lively, varied prose.
For engineers
Maintains k partial hypotheses ranked by cumulative log-prob, expanding and pruning to k each step until end tokens fire.
Related
- Greedy decoding — The simpler one-path cousin.
- Sampling — What replaced it for open-ended text.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome