Continual Learning
Also called Lifelong Learning
Teaching a model new things over time without making it forget what it already knew.
Think of it like
Adding new recipes to a chef’s repertoire without them suddenly forgetting how to make the old ones.
Example
A support model needs this quarter’s product updates added on top, but must keep answering last year’s questions correctly.
How it actually works
The core tension is stability versus plasticity: learn new information fast enough to stay current, but not so aggressively that you overwrite old knowledge — the dreaded catastrophic forgetting. Techniques include replaying old data, freezing parts of the network, and adding small adapters. Truly seamless continual learning remains an open problem.
For product teams
The dream of updating a model with fresh knowledge without a full, expensive retrain — with real trade-offs today.
For engineers
Sequential training on new data while preserving prior performance, balancing plasticity against forgetting.
Related
- Catastrophic forgetting — Fights catastrophic forgetting.
- Replay Buffer — Often uses a replay buffer.
- Continued pretraining — Overlaps with continued pretraining.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome