Teacher model
A big, capable model whose outputs are used to train a smaller one.
Think of it like
Like an expert who writes out worked solutions so an apprentice can learn from them.
Example
A large frontier model labels a million examples, and a compact model trains on those labels to imitate it at a fraction of the size.
How it actually works
In distillation, the teacher provides targets — final answers or full probability distributions — richer than raw labels, letting the student absorb its behavior. The student can approach the teacher’s quality on many tasks while being far cheaper to run, though it inherits the teacher’s blind spots too.
For product teams
How you get near-frontier quality at small-model cost — with the teacher’s biases along for the ride.
For engineers
The source model in distillation, supplying soft targets or labels the student is trained to match.
Related
- Student model — The smaller model it trains.
- Frontier model — A frequent teacher.
- Small language model — What small models it produces.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome