ROUGE
A metric that counts overlapping words between generated and reference text—faster and more intuitive than BLEU, standard for summarization.
Think of it like
Like checking if a book summary mentions the key characters and plot points.
Example
Generated summary: 'Alice went to the store and bought apples.' Reference: 'Alice bought apples at the store.' ROUGE-1 (unigram overlap) scores high.
How it actually works
ROUGE has variants: ROUGE-1 (unigram), ROUGE-2 (bigram), ROUGE-L (longest common subsequence). It's better than BLEU for synonyms and paraphrases. Still doesn't measure meaning, but more robust. Standard for summarization eval. Human eval on a subset always recommended.
For product teams
ROUGE is a reasonable automated metric for summarization and short text generation. Supplement with human eval.
For engineers
Report ROUGE-1, ROUGE-2, and ROUGE-L; understand they're proxies, not gospel.
Related
- BLEU — Predecessor metric.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome