Truncation
Chopping text to fit a length limit — usually by dropping whatever spills over the edge.
Think of it like
Like trimming a too-long photo to fit the frame; whatever sticks out gets cut off.
Example
A long transcript exceeds the context window, so the pipeline truncates the oldest lines, and the model never sees how the meeting began.
How it actually works
Truncation happens on input (to fit the context window) or output (to hit max tokens). The danger is silent information loss: if you cut the wrong end you can lop off the instructions or the crucial early facts, so which side you trim from matters as much as that you trim.
For product teams
A quiet source of “why did it ignore that?” bugs — the model may simply never have seen the cut text.
For engineers
Drops tokens beyond a length budget from a chosen end (left/right) before or after generation.
Related
- Max tokens — The output limit that forces it.
- Context window — The input budget it protects.
- Padding — Its opposite: filling short inputs.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome