Explain like I'm five · AI

What is Bias?

AI bias, told as a story about a parrot whose book pile leans one way — so its answers lean the same way.

draw a doctor — and a nursewhat the parrot drewdoctor → always the same looknurse → always the other lookRESUMEname: Ascore 90RESUMEname: Bscore 60same resume, only the name differsalways the same look, a different score for a different name — something leans

Every doctor looks the same; a different name gets a different score.The parrot's answers lean one way. Why?

a book pile leaning one wayit picks the bean it saw mostCOUNTED IN THE BOOKSdoctor = a man …… 1000×doctor = a woman …… 10×→ next bean: a mannot the parrot's fault — the books are like thatit picks the most frequent bean — leaning books make a leaning parrot

The parrot picks the most frequent bean — leaning books make a leaning parrot.The parrot picks what it saw most in its books. If the books tell one side far more often, only that side comes out. It isn't the parrot's fault.

the leaning pilea doctor is… a man!examinerMIRRORa mirror of the books — and usbias is the lean of the books coming straight out in the answers

Bias is the lean of the books coming straight out in the answers.Not the parrot's fault — it mirrors the books, and us, because people wrote them. So instead of erasing it, we measure it, reduce it, and let a person look at what matters.

Put it in the test bookquestions that measure the lean
Swap just the namegender, region too — same answer expected
People decide what mattershiring, loans, care
Manners lessons reduce itless, never zero
only the name differsRESUME 1name: Minjun5 yearslicensed ✓RESUME 2name: Aisha5 yearslicensed ✓90 points60 pointsSCORE SHEETname swappedscore gap ×the examiner flags itsame resume, different name, different score — so we swap and measure

Feed the same resume with only the name changed; if the score moves, flag it.Change one thing — name, gender, region — and keep everything else the same. If the answer changes, it leans. The examiner's score sheet keeps such questions and measures them again and again.

TEST BOOKsame answer, new name ✓same answer, new region ✓same answer, new gender ✓measure, then reduceit flips even the factsnow it leans the other waya new lean from the fix — over-correctionit never fully goes away — keep measuring, and a person decides what matters

Fixing it can tip it the other way. So we keep measuring rather than declaring it gone.Manners lessons reduce the lean, but too much and it bends the facts (over-correction). That is why hiring, loans and the like end with a person looking last.

In one breath

Bias = the lean of the books the parrot read coming straight out in its answers. It mirrors the books and us rather than being the parrot's fault, so we keep measuring and let people make the decisions that matter.

When the training data's distribution skews toward certain groups or views, the model's output skews the same way. It is measured with counterfactual tests (change only the name, gender or region) and fairness benchmarks, reduced through alignment (RLHF and the like) while watching for over-correction, and high-stakes decisions require a human check.

When grown-ups say it

Bias
The lean in the books. It comes straight out in the answers — a mirror, not the parrot's fault.
Training data bias
The pile leans one way. One side appears far more often, so only that side comes out. The pile itself: → the reading school
Fairness evaluation
The lean questions in the test book. Measures whether answers differ by group. → the examiner's score sheet
Counterfactual test
Swap just the name. Change one thing — name, gender, region — and see if the answer stays the same.
Representation
Is everyone in the books evenly? Groups that appear rarely the parrot knows poorly, or draws one way only.
Over-correction
Tipping the other way while fixing. It starts bending facts. So after a fix, measure again.
Alignment
The parrot's manners lessons. Reduces the lean, never removes it. → the parrot's manners lessons
Human review
A person decides what matters. Hiring, loans and care never end on the parrot's answer. → a person goes last
Next story
The Secret Beans We Gave the Parrot →
Want another term as a picture book? Request →