Prompt compression
Reducing a long prompt or context to its essential meaning — preserving information while shrinking token count.
Think of it like
Summarizing a 50-page report into a 5-page executive summary.
Example
A 500-token document compressed to 200 tokens via summarization or selector techniques, then used in a prompt.
How it actually works
Compression can be lossy (summary) or lossless (clever encoding). Lossy is simpler but risks missing details. Lossless is hard (compression algorithms don't beat LLMs at understanding semantics). Some models specifically trained for compression exist. Metrics: information retention vs. token savings.
For product teams
Reduces cost for long-context tasks.
For engineers
Experiment with summarization + compression. Evaluate retention with eval set.
Related
- Context window management — Solves context window management challenges.
- Token optimization — Overlaps with token optimization.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome