Test-Time Compute
Spending more computing power at the moment of answering — not during training — to get a better answer.
Think of it like
Like giving yourself more time on an exam question instead of studying harder beforehand — more thinking now, right when it counts.
Example
A reasoning model that takes 30 seconds and writes pages of internal steps for a hard proof is burning test-time compute.
How it actually works
For years, gains came from bigger models and more training. Test-time compute flips some of that: keep the model fixed but let it think longer, sample more candidates, or search over options at answer time. On hard problems this buys real accuracy, but it directly raises latency and cost per request, so it’s a dial to spend deliberately.
For product teams
A new cost lever — you can now trade money-per-answer for quality, live.
For engineers
Allocating additional inference-time computation (longer generations, sampling, search) to improve outputs.
Related
- Inference-Time Scaling — A near-synonym emphasizing the scaling angle.
- Reasoning Model — The model type built around it.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome