Capacity
How much a model can potentially learn — roughly, how complex a pattern it can represent.
Think of it like
Like the size of a suitcase: a bigger one holds more, but an oversized bag half-empty just gets messy and heavy.
Example
A tiny model can’t capture the nuance of legal text — too little capacity — while a huge one memorizes the training set instead of generalizing.
How it actually works
Capacity grows with parameters, depth, and width, and it sets the ceiling on complexity a model can fit. Too little and you underfit; too much for your data and you overfit or waste compute. The art is matching capacity to the amount and richness of data, which is exactly what scaling laws try to quantify.
For product teams
Bigger isn’t automatically better — capacity has to be matched to the data and the task.
For engineers
Informal measure of a model’s expressive range, scaling with parameter count, depth, and width.
Related
- Overfitting — Too much for the data leads here.
- Underfitting — Too little leads here.
- Parameters — The knob that raises it.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome