BM25
A classic keyword-ranking formula that scores documents by how well their words match the query.
Think of it like
A smart word-counter — rewards documents that use your rare search terms often, but doesn’t double-count common filler.
Example
Searching “quarterly revenue Anthropic,” BM25 ranks highest the docs that use those exact, uncommon words most.
How it actually works
BM25 is decades old and still a workhorse: it scores on term frequency (how often a query word appears) balanced against how rare that word is overall and how long the document is. It’s fast, needs no training, and is unbeatable at matching exact terms — names, codes, jargon — which is precisely where embeddings struggle. That durability is why nearly every hybrid search still has BM25 doing the keyword half.
For product teams
The old keyword approach still beats fancy embeddings on exact terms — don’t retire it, combine with it.
For engineers
A sparse ranking function scoring on term frequency, inverse document frequency, and length normalization.
Related
- Semantic Search — The meaning-based approach it’s paired with.
- Hybrid Search — Where it’s combined with vectors.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome