Explain like I'm five · AI Theory

What is an Adversarial Attack?

Adversarial attacks, told as a story about produce that looks perfectly fine to a person but is read entirely differently by the sorter.

the person who added the dotsthat's a rotten one ×to a human eye it's a perfectly fine apple — with a few nearly invisible dots mixed in

To a human eye it's a fine apple — but the sorter reads it as rotten.Just a handful of nearly invisible dots have been mixed into the apple's photo.

reads by adding up pixel numberssees the whole shape at a glancenudge the pixel numbers ever so slightly, and the sorter's classification can flip

The sorter reads numbers; a person reads shape.Nudge the photo's pixel numbers ever so slightly, and the sorter's result can flip.

what a person seesjust an applewhat the sorter readsREJECT BINthis is a reject ×an adversarial attack subtly perturbs the input so classification flips, though a person sees no difference

A normal apple to a person — but the sorter classifies it as a reject.The input photo is almost unchanged, yet the sorter's call flips from pass to reject.

Add a noise patternscatter dots too faint to notice
Nudge the edgespush the shape's edges a little
Stick on a patterneven fools the sorter when stuck on real produce
Search by trialadjusting the perturbation until the sorter errs
original producePASS BINsubtly perturbed produceREJECT BINidentical to a human eyeadd just a handful of dots, and the produce lands in a different bin

Add just a handful of dots, and the produce lands in the reject bin instead of the pass bin.Two items identical to a human eye — yet the sorter passes one and rejects the other.

mixes rotten items into the seed (training data)→ attacks during teachingholds a perturbed item up to a sorter that's already fully trained→ attacks at the moment of judgingspoiling the seed is data poisoning — fooling the trained sorter is an adversarial attack

Data poisoning and adversarial attacks differ in timing — during teaching, or at the moment of judging.Spoiling the seed is data poisoning; fooling the already-trained sorter is an adversarial attack.

In one breath

Adversarial attack = subtly perturbing an input so it looks the same to a person but the sorter classifies it completely differently.

An adversarial attack is an inference-time attack that adds a perturbation too small for a human to notice (noise) or a pattern a human would shrug off (a sticker), making an already-trained classifier misjudge exactly the way the attacker wants. White-box attacks, which need knowledge of the model's weights, make a more precise perturbation; black-box attacks, without that knowledge, work by repeated trial. The same principle applies beyond images — to audio, text, and tabular classifiers alike. Unlike data poisoning, which corrupts the training data itself, an adversarial attack targets a model that has already finished learning.

When grown-ups say it

Adversarial attack
A perturbation that looks the same to a person but gets classified differently by the sorter. Happens against a model that has already finished training.
Data poisoning
Spoiling the seed — the training data itself. A different attack timing — during training. → the LLM-security field's data poisoning
Perturbation
A small change too faint to notice. A slight nudge to dots or edges.
White-box · black-box
Whether you know the sorter's inner rule, or only try by trial. Knowing the inner rule lets the perturbation be far more precise.
Misclassification
Dropping an item into the wrong bin. The outcome an adversarial attack is built to cause.
Confusion matrix
The four-box table sorting pass/fail stamps. Mixed-in adversarial attacks raise the counts in the mistake boxes (FP, FN). → The Four-Box Stamp Ledger
Outlier
The one item that stands out from the rest. One defense against adversarial attacks is spotting inputs that stand out. → The Produce That Stands Out from the Crowd
Image injection (LLM/vision angle)
A related attack that fools an LLM with a command hidden in a photo. The angle where an LLM reads a hidden command in a photo lives in the LLM-security field. → The Picture Order That Fools the Eye
Next story
The Four-Box Stamp Ledger →
Want another term as a picture book? Request →