Draft Model
The small, fast model that proposes candidate next words for a bigger model to check.
Think of it like
Like an intern who quickly sketches the next few lines so the expert only has to review, not write from scratch.
Example
In speculative decoding, a 1-billion-parameter draft model guesses tokens that a 70-billion-parameter model then verifies.
How it actually works
It needs to be fast and roughly agree with the big model, since every token the big model rejects is wasted draft work. The closer the draft’s guesses match the target, the more speedup you get. Choosing or training a good draft model — fast enough to help, aligned enough to be accepted — is the core engineering knob of speculative decoding.
For product teams
The cheap helper that makes the expensive model respond faster.
For engineers
A lightweight model proposing tokens for parallel verification by the target model.
Related
- Speculative Architecture — The technique it powers.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome