Data Poisoning
Also called Data Poisoning
Sneaking bad examples into training data so the model learns a hidden flaw or backdoor.
Think of it like
Slipping a few doctored flashcards into a student’s deck so they confidently learn one wrong answer.
Example
An attacker seeds the web with pages pairing a rare trigger phrase with malicious behavior, betting the phrase gets scraped into the next training run.
How it actually works
Models learn whatever patterns are in their training data, and much of that data is scraped from the open internet — which anyone can write to. Poisoning plants targeted examples to install a backdoor (behave normally until a trigger appears) or to skew the model’s beliefs. It’s hard to detect because the corrupted behavior hides until activated, and hard to prevent at web scale. Data provenance and filtering are the main defenses.
For product teams
A supply-chain risk in the training pipeline — the threat is upstream of your product, in the data itself.
For engineers
Adversarial examples injected into training corpora to install backdoors or bias; countered with provenance controls and data filtering.
Related
- Model Extraction — A model-stealing cousin of adversarial attacks.
- Adversarial Example — Crafted inputs that fool a trained model.
- AI Safety — Part of the AI safety landscape.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome