Synthetic Bootstrapping
Use a model to generate its own training data, then train on it to get better at generating more.
Think of it like
Pulling yourself up by your bootstraps — a small start that generates the fuel for a bigger climb.
Example
A model writes thousands of math solutions, you keep only the ones that check out, and fine-tune on those to boost it.
How it actually works
Starting from a small seed of quality data or a decent base model, you generate new examples, filter them for correctness, and train on the survivors — then repeat, each round raising the floor. It works best when you can cheaply verify outputs (math, code, tests). The danger is compounding the model’s own errors when your filter is weak.
For product teams
A way to grow training data without armies of human labelers — as long as you can check quality automatically.
For engineers
Iterative self-generation of training data with a verification filter to bootstrap capability gains.
Related
- Synthetic data — Produces synthetic data.
- Rejection sampling — Filtered by rejection sampling.
- Self-distillation — A cousin of self-distillation.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome