Token optimization
Minimizing token usage — both input and output — to reduce latency and cost while maintaining quality.
Think of it like
Writing a haiku instead of a sonnet: same message, fewer words.
Example
"What is 2+2?" (4 tokens) vs. "Please tell me the result of 2+2?" (8 tokens). Same answer, 50% fewer tokens.
How it actually works
Token count scales with prompt length and model output length. Optimization strategies: (1) Compress prompts, (2) Use shorter examples, (3) Limit max output tokens, (4) Cache prompts (if API supports it), (5) Choose smaller models when possible.
For product teams
Direct cost savings. Faster responses.
For engineers
Profile token usage. Identify bottlenecks. Compress, cache, or use smaller models.
Related
- Context window management — Part of context window management.
- Prompt compression — Related to prompt compression.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome