Compute-optimal
Also called Compute-Optimal Training
Splitting a fixed compute budget between model size and data so you get the lowest loss for the money.
Think of it like
With a set grocery budget, choosing the mix of ingredients that makes the best meal — not just the biggest one.
Example
Given a fixed number of GPU-hours, a team picks a smaller model trained on more tokens because the math says it lands lower loss than a bigger, under-fed one.
How it actually works
For a fixed compute budget, there is a sweet spot in how you trade parameters against training tokens. The Chinchilla work showed many large models were over-parameterized and under-trained — too big for the data they saw. Compute-optimal means sitting at that frontier, though it optimizes training cost, not inference cost, so deployment concerns can push you smaller still.
For product teams
It reframed "bigger is better" into "right-sized for the data" — and made many models cheaper to serve.
For engineers
Allocate a fixed FLOPs budget between N (params) and D (tokens) to minimize loss, per Chinchilla-style scaling.
Related
- Chinchilla scaling — The result that defined this sweet spot.
- Scaling laws — The curves this optimizes along.
- Tokens per parameter — The ratio at the heart of the trade-off.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome