Token limit
The ceiling on how many tokens a model can handle at once, input and output combined.
Think of it like
Like a desk that only fits so many papers — pile on more and the earliest ones slide off.
Example
A user pastes a 200-page PDF into a chat with a 128k-token limit and gets an error because the document alone blows past the ceiling.
How it actually works
The token limit is the physical size of the context window; prompt, history, and generated reply all draw from the same pool. Exceeding it forces truncation or an error, and even well under it, models often use the far ends of that budget less reliably than the edges.
For product teams
Shapes what you can feed a model in one go and quietly caps how much history a chat can keep.
For engineers
Max total tokens (prompt + completion) the model accepts; overflow triggers truncation or a hard error.
Related
- Context window — The window this limit measures.
- Truncation — What happens on overflow.
- Lost in the middle — Usable budget shrinks toward the middle.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome