Model inversion
Also called Model Inversion Attack
Reconstructing chunks of the training data by interrogating the model that learned from it.
Think of it like
Reassembling a shredded document by studying the outline it left pressed into the notepad beneath.
Example
By querying a face-recognition model repeatedly, an attacker recovers a blurry but recognizable image of someone in its training set.
How it actually works
The attacker optimizes an input to maximize the model’s confidence for a target label, effectively running training in reverse to surface a representative example. It works best on overfit models that memorized specifics. It sits alongside membership inference as proof that models can leak the data inside them.
For product teams
A model can quietly become a copy of the sensitive data it was trained on — a real exposure risk, not a hypothetical.
For engineers
Gradient-based optimization against class confidence recovers prototypical inputs; defenses include DP, output truncation, and less overfitting.
Related
- Membership inference — A sibling privacy attack that only asks "was this record in?".
- Motivates differential privacy in training.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome