Recursive Chunking
Also called Recursive Splitting
Splitting text by trying big separators first — paragraphs, then sentences — until pieces fit.
Think of it like
Breaking a chocolate bar along the deepest grooves first, snapping smaller only if the piece is still too big.
Example
A splitter tries paragraph breaks, and only where a paragraph is too long does it drop to sentence-level cuts.
How it actually works
Recursive chunking uses an ordered list of separators — say, double newlines, then newlines, then spaces — and applies each in turn until chunks are under the size limit. This keeps semantically coherent units together when possible and only makes ugly cuts as a last resort. It is the default in many frameworks because it is simple and respects document structure better than fixed slicing.
For product teams
A sensible default that keeps chunks coherent without much tuning.
For engineers
Hierarchical separator list applied top-down until the size constraint is met; balances structure and length.
Related
- Chunking — A strategy within chunking.
- Sentence Splitting — Respects natural structure like sentence splitting.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome