Synthetic data
Training data made by a model rather than collected from the real world.
Think of it like
Like a flight simulator generating endless practice scenarios so pilots don’t need a real storm to train on.
Example
A strong model writes thousands of math problems with worked solutions, and those become training data for a smaller model.
How it actually works
Synthetic data lets teams manufacture examples where real data is scarce, private, or expensive, and to target specific weaknesses. Used well, it boosts capability cheaply; used carelessly, it risks model collapse — training on a model’s own averaged output can erode diversity and amplify its errors over generations. Quality control and grounding in real data are the guardrails.
For product teams
A powerful way to cover gaps and rare cases — but data made by models can quietly inbreed if you’re not careful.
For engineers
Model-generated training examples; scalable but risks distribution collapse without real-data anchoring/filtering.
Related
- Supervised fine-tuning — A source for supervised fine-tuning.
- Teacher model — Often produced by a teacher model.
- Data augmentation — A form of data augmentation.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome