Retrieval evaluation
Measuring whether the documents you pulled actually contain the answer to the question.
Think of it like
Fact-checking the articles your research assistant grabbed — did they grab the right ones?
Example
You ask a RAG system "What does the company spend on R&D?" and it pulls five documents. Three are about marketing, two are about finance. Only the finance ones help — that's a retrieval failure.
How it actually works
Retrieval is where RAG lives or dies. You measure it with metrics like precision (fraction of retrieved docs that were relevant), recall (fraction of relevant docs that were actually retrieved), and MRR (mean reciprocal rank — how high up was the first good one). Bad retrieval kills everything downstream, no matter how good your language model is.
For product teams
If your RAG system gives wrong answers, check retrieval first — it's often the culprit, and fixing it beats retraining the model.
For engineers
Evaluate retrieval separately from generation using metrics like precision@k, recall@k, NDCG, and MRR against gold documents.
Related
- Part of the RAG pipeline.
- Partner metric for generation quality.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome