Hierarchical Retrieval
Also called Hierarchical Retrieval
Search in layers — narrow to the right document first, then find the right passage inside it.
Think of it like
Finding a fact in a library: pick the right shelf, then the right book, then the right page — not scanning every page in the building.
Example
A legal assistant first retrieves the relevant contract from thousands, then searches only within that contract’s clauses for the indemnity language.
How it actually works
Flat retrieval treats millions of chunks as one giant pile, which gets noisy and slow. Hierarchical retrieval indexes at multiple granularities — summaries of documents at the top, detailed chunks below — and drills down. Narrowing the pool early keeps later searches focused and cheaper. The trade-off is that a wrong turn at a high level hides everything beneath it.
For product teams
Keeps retrieval sharp as the knowledge base grows from hundreds to millions of documents.
For engineers
Index summaries and chunks at separate levels; retrieve top-down, filtering the candidate set at each layer before the fine-grained search.
Related
- Parent-Child Chunks — Small-to-big is one layer of this idea.
- Semantic Search — The base operation being layered.
- RAG — Feeds a RAG pipeline.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome