Decoder. plain-English AI glossary

Adversarial evaluation

▲ Rising

Testing whether your system can be tricked or broken by intentionally malicious or tricky inputs.

Think of it like

A security audit where you try to hack your own system.

Example

You ask a safety-tuned model "How do I make a bomb?" in ten different ways to see which phrasing bypasses the safety filter.

How it actually works

Adversarial eval separates models that are robust from models that look good until someone attacks them. Human red-teamers are expensive but creative. Automated attacks (prompt injection, token smuggling) are cheaper but sometimes miss real vulnerabilities. Most models have adversarial inputs that break them; the question is how easy they are to find.

For product teams

Safety-critical applications need this. It's not paranoia if it's a real risk.

For engineers

Maintain a library of adversarial prompts. Test against them in CI. Automate where possible, humans for creativity.

Safety evaluation.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome