Checkpoint Averaging
Average several saved snapshots from a single training run into one steadier final model.
Think of it like
Blending a few photos of a moving subject to cancel the blur and get one sharp shot.
Example
Rather than shipping the last step, a team averages the final five checkpoints and gets a small, free quality gain.
How it actually works
Late in training the weights oscillate around a good region rather than settling exactly. Averaging several nearby checkpoints lands you in a flatter, more robust center of that region, which usually generalizes a touch better than any single snapshot. It’s a standard, near-zero-cost finishing move.
For product teams
A no-downside trick to slightly improve the final model from runs you already completed.
For engineers
Averaging parameters of consecutive checkpoints from one run to reduce oscillation-driven variance.
Related
- Weight Averaging — A specific use of weight averaging.
- EMA — The streaming version is EMA.
- Checkpoint — Operates on saved checkpoints.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome