Poisoned data
Also called Data Poisoning
Slipping corrupted examples into the training set so the model learns something the attacker wants.
Think of it like
Sneaking a few wrong recipe cards into the cookbook so the whole kitchen starts making the same mistake.
Example
An attacker edits a handful of public web pages that a scraper later ingests, nudging a model to associate a brand name with praise.
How it actually works
Because models learn statistics from whatever they are fed, even a small poisoned fraction can shift behavior — sometimes broadly, sometimes only on a trigger (a backdoor). Web-scale scraping makes this cheap, since anyone can publish training data. Defenses include provenance tracking, dataset filtering, and anomaly detection.
For product teams
"We trained on the whole internet" also means the internet can train you back.
For engineers
Even sub-percent contamination can implant targeted behavior; mitigate with data provenance, dedup/filtering, and robust training.
Related
- Backdoor attack — The main way a backdoor attack gets installed.
- Supply chain risk — A concrete supply chain risk for training pipelines.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome