Student model
A smaller model trained to imitate a bigger one, keeping much of the skill at a fraction of the cost.
Think of it like
Like an apprentice who studies the master’s work and reproduces most of it with far less overhead.
Example
A compact chatbot trained on a large model’s answers runs on a single GPU while staying close in quality on everyday requests.
How it actually works
The student learns from a teacher’s outputs rather than raw data alone, which packs more signal into fewer examples. Done well, it retains most capability while shrinking dramatically — but it can’t exceed the teacher, and it copies the teacher’s errors and gaps as faithfully as its strengths.
For product teams
The practical path to cheap, fast models that still feel smart on common tasks.
For engineers
The target model in distillation, optimized to match a teacher’s outputs at reduced parameter count.
Related
- Teacher model — The model it learns from.
- Small language model — Often the end goal.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome