Max tokens
A hard cap on how many tokens the model is allowed to generate before it must stop.
Think of it like
Like a word limit on an essay — hit it and the pen gets taken away mid-sentence if needed.
Example
An app sets max tokens to 300 for summaries; a user asks for a full report and the answer cuts off abruptly at the ceiling.
How it actually works
It bounds only the output, not the input, and it’s a budget and safety lever — capping cost and latency. The catch is that the model doesn’t “aim” to finish within it; if the cap is too low the response just truncates mid-thought, so it should be sized to the task.
For product teams
Your main control on cost and response length, but set it too tight and answers get chopped.
For engineers
Upper bound on generated tokens; generation halts at this count or at a stop condition, whichever comes first.
Related
- Context window — Counts against the shared budget of the context window.
- Stop sequence — The other way generation ends.
- Truncation — What gets cut when the cap is hit.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome