Adversarial Example
Also called Adversarial Example
An input tweaked in a way humans barely notice but that makes the model badly wrong.
Think of it like
An optical illusion tuned for a machine — a tiny sticker on a stop sign that makes a car read it as "speed limit 45."
Example
A few carefully chosen pixels added to a panda photo, invisible to you, make an image classifier label it a gibbon with high confidence.
How it actually works
Models draw decision boundaries in high-dimensional space, and those boundaries have exploitable seams. An adversarial example nudges an input just across a boundary — often imperceptibly — to force a confident wrong answer. It started in image classifiers but applies to text and audio too. It reveals something uncomfortable: high accuracy on normal inputs doesn’t mean the model is robust to inputs an adversary designed on purpose.
For product teams
A reminder that "works great in the demo" and "safe against attackers" are different guarantees.
For engineers
Small, often imperceptible perturbations that cross a decision boundary; robustness requires adversarial training or defenses, not just clean accuracy.
Related
- Jailbreak — Text-domain attacks share the flavor.
- Data Poisoning — Poisoning plants such examples in training.
- Red Teaming — Red teaming hunts for these weaknesses.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome