Decoder. plain-English AI glossary

Robustness testing

▲ Rising

Systematically changing inputs slightly (typos, paraphrases, noise) to see whether outputs stay consistent.

Think of it like

A wine tasting where you test whether the wine's quality is consistent even when slightly off.

Example

You take your test set and add typos, reorder sentences, and change synonyms. If your model's answers change drastically, it's not robust.

How it actually works

Robustness testing catches models that are brittle — good on the exact examples they've seen, bad on anything slightly different. It's cheaper than adversarial eval but less thorough. You can automate it by programmatically perturbing your test set.

For product teams

Tell you whether improvements are real or just overfitting to your eval set.

For engineers

For each example, generate perturbations (typos, synonym replacements, paraphrases). Check if outputs match via embedding similarity.

Generalization metric.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome