Sequence length
How many tokens are actually in a given input or output right now.
Think of it like
The number of cars in the train you’re running today, versus how long the platform can hold.
Example
A 500-token prompt has a sequence length of 500, well under a 128K context limit.
How it actually works
Sequence length is the count of tokens being processed in a specific pass, while context length is the ceiling. Attention cost grows roughly with the square of sequence length, which is why long inputs get expensive fast. Keeping sequences tight is a real performance lever.
For product teams
Shorter sequences mean lower latency and cost — trimming inputs is a cheap optimization.
For engineers
The token count n of a given batch element; attention is O(n²) in it.
Related
- Context length — Capped by context length.
- Tokens — Counted in tokens.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome