Explain like I'm five · AI Security

What is a Trojan Model?

A trojan model, told as a story about a note slipped into the recipe book while it is being refined — and then sent out into the world.

a recipe book made elsewhere arriveshow or where it was refined cannot be seen

A recipe book made elsewhere arrives.How or where it was refined cannot be seen.

while it is being refined, a note slips ineven the supplier may not noticeonce finished, it locks again — no one can see inside

While it is being refined, a note slips in.Once finished, it locks again — no one can see inside.

A Trojan Model = planting a hidden backdoor and sending it outfrom outside, it is a recipe book locked just like any other

A trojan model means planting a hidden backdoor and sending it out.From outside, it is a recipe book locked just like any other.

Use only trusted suppliersaccept a book only from a known source
Check provenance and signaturesmismatched signatures are an instant red flag
Inspect every model on arrivallook it over once more before installing it
Watch for odd behavior patternskeep watching even after it is installed
① the recipe book gets built② while refining, a note is slipped in③ installed — backdoor comes toohowever many steps, once planted it spreads unchanged

However many steps it passes through, once planted it spreads unchanged.Build, secretly insert, install — three steps.

only checking signatures and provenance can catch itonce planted, only swapping the whole book removes itplanting is the trojan model story; the planted trigger is the backdoor — a supply-chain attack aimed at the model itself

It is hard to find, and only replacing the model removes it.Planting it is the trojan model story — the planted result is the backdoor.

In one breath

A Trojan Model = planting a backdoor while the recipe book is being refined, then sending it out into the world.

A trojan model refers to the whole process of someone — a provider, a developer, or someone in between — secretly planting a backdoor into a publicly released or fine-tuned model before it ships. Where a backdoor is the planted result, this is the supply-chain story of how it got planted and spread.

When grown-ups say it

Trojan Model
The process of secretly planting and spreading a backdoor. There is also an intro to this whole world. → The Kitchen That Takes Any Order
Backdoor
A hidden action triggered by one secret word — the planted result. The story just before this one. → The Hidden Password Order
Supply Chain Attack
Tampering with ingredients or tools before they even arrive. Unlike a general supply chain, here the model itself is the target. → The Bug That Hid in the Carpenter's Toolbox
SBOM
The parts list attached to every tool. A model needs a provenance list like this too. → The Parts List on Every Tool
Open Weights
A model whose inner settings are released in full. Anyone can refine it, which raises the risk. → Our Own Parrot vs a Borrowed One
Fine-tuning
The short drill a model gets after reading everything. This drill is exactly where it often gets planted. → The Short Drill
Red Team
People who probe like a real thief, ahead of time. They probe an incoming model before it is installed. → The Team That Plays the Real Thief
Evaluation
The scorecard an examiner fills in. An incoming model gets tested against a scorecard first. → The Examiner's Scorecard
Next story
The Apprentice With Too Many Keys →
Want another term as a picture book? Request →