Trojan
Also called Trojaned Model
A hidden malicious behavior baked into a model, dormant until its secret trigger wakes it up.
Think of it like
A sleeper agent who lives an ordinary life until they hear the one code phrase.
Example
A team downloads a popular open-weights checkpoint that quietly emits attacker-chosen text whenever a specific rare token appears.
How it actually works
"Trojan" usually names the payload — the planted behavior — where "backdoor" names the mechanism that installs and triggers it; people use them almost interchangeably. Because the model looks clean on normal inputs, trojans survive standard testing. The defense is trust in the supply chain plus targeted trigger-hunting.
For product teams
It is the reason you cannot fully trust a model just because it behaves well in demos.
For engineers
The dormant payload activates on a trigger distribution absent from your eval set; scan weights and probe with synthesized triggers.
Related
- Backdoor attack — The delivery-and-trigger mechanism behind it is a backdoor attack.
- Model provenance — A reason to verify model provenance.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome