Backdoor attack
Also called Neural Backdoor
Secretly training a model to behave normally until it sees a hidden trigger, then misbehave on command.
Think of it like
A vending machine that works fine for everyone, but dispenses free product to whoever taps a secret knock.
Example
A model classifies images correctly until a tiny yellow sticker appears in the corner, which the attacker trained as a switch to force a wrong label.
How it actually works
The attacker poisons a slice of training data so the model links a rare trigger pattern to an attacker-chosen output, while accuracy on clean inputs stays high enough to pass review. That normal-looking behavior is what makes backdoors hard to catch. It usually rides in through poisoned data or a tampered pretrained checkpoint.
For product teams
A model can pass all your evals and still carry a hidden switch, which is why model provenance matters.
For engineers
Trigger-conditioned poisoning creates a spurious feature-to-target link; detect via activation clustering, trigger reconstruction, or fine-tuning on clean data.
Related
- Trojan — The planted mechanism is often called a trojan.
- Poisoned data — Delivered through poisoned data.
- Model provenance — Raises the stakes on model provenance.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome