Decoder. plain-English AI glossary

MT-Bench

▲ Rising

A benchmark of 80 multi-turn conversations that test dialogue quality, reasoning, and instruction-following across 8 domains.

Think of it like

Like a Turing test where a judge rates the quality of a back-and-forth conversation.

Example

A user asks a question, the model answers, the user asks a follow-up, and raters evaluate whether the model's responses were coherent and helpful.

How it actually works

MT-Bench uses LLM-as-judge (GPT-4) to evaluate responses, so it's fast but not perfect. Useful for detecting regressions and comparing models, but human eval on a subset is still recommended. Multi-turn helps catch consistency issues—does the model remember context?

For product teams

MT-Bench gives a quick sense of dialogue quality. Useful for A/B testing prompt changes.

For engineers

Supplement with human eval; don't rely solely on LLM judges.

Benchmark — Category.
LLM-as-Judge — Judge it uses.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome