BLEU
A metric that measures how much generated text overlaps with reference text—once the standard for machine translation, now mostly legacy.
Think of it like
Like checking if a student's essay quotes match the required source material.
Example
Model generates: 'The cat sat on the mat.' Reference: 'The cat sat on the mat.' BLEU scores high due to n-gram overlap.
How it actually works
BLEU counts matching n-grams (sequences of n words) between generated and reference text. It's fast and language-agnostic, but it doesn't understand meaning. A paraphrase with different words gets low BLEU even if it's better. Research has largely moved to ROUGE and semantic metrics because BLEU doesn't correlate well with human judgment.
For product teams
BLEU is outdated for modern NLG tasks. Don't ship decisions based on BLEU scores.
For engineers
Use ROUGE, semantic similarity, or human eval instead.
Related
- ROUGE — Modern replacement.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome