Model Extraction
Also called Model Extraction
Cloning a model’s behavior by querying it a lot and training a copy on its answers.
Think of it like
Reverse-engineering a secret recipe by ordering the dish hundreds of times and tasting your way to a replica.
Example
A competitor sends millions of prompts to a paid API, collects the outputs, and trains a cheaper knock-off that mimics the original.
How it actually works
You don’t need the weights to approximate a model — its inputs and outputs are a teaching signal. By querying systematically and training a student model on the responses (a form of distillation), an attacker can recover much of the behavior without the training cost. Defenses include rate limits, watermarking outputs, and detecting extraction-shaped query patterns, but none are airtight against a patient adversary.
For product teams
An IP and moat risk — your model’s outputs are, in effect, a leaky blueprint of it.
For engineers
Query-based behavior cloning via output distillation; mitigated with rate limiting, output watermarking, and anomaly detection.
Related
- Watermarking — Watermarks help trace a stolen copy.
- Data Poisoning — A poisoning cousin among training attacks.
- AI Safety — Sits within AI security concerns.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome