Emergence
Also called Emergent Behavior
When a capability appears fairly suddenly at a certain scale, seeming absent in smaller models and present in bigger ones.
Think of it like
Water that just sits there as you heat it — until it hits a boil and abruptly changes state.
Example
Small models score near-random on multi-step arithmetic, then past a size threshold they start getting it right, seemingly out of nowhere.
How it actually works
As models scale, some skills improve smoothly while others look like sharp jumps. Debate continues over how "real" the sharpness is — some apparent leaps flatten out under smoother metrics, suggesting the jump is partly an artifact of harsh pass/fail scoring. Either way, it complicates planning: scaling laws predict loss well but not exactly when a given ability shows up.
For product teams
Why you cannot always promise a feature at a target size — some abilities only show up when they show up.
For engineers
Capabilities that appear abruptly with scale; partly metric-dependent and not directly forecast by loss curves.
Related
- Scaling laws — Predicts loss but not when abilities appear.
- Emergent ability — The broader notion of scale-unlocked skills.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome