Explain like I'm five · AI Theory

What is an Unbiased Estimator?

Unbiased estimators, told as a story about a scale that may miss once, but whose average lands on the true weight over many weighings.

can't weigh the whole truck — guess the average from one boxcan one box's weight be trusted for the whole truck's average?

Unable to weigh the whole truck, the inspector weighs just one box.Can one box's weight be trusted for the whole truck's average?

+100gagain +100gno matter how many times you weigh, this scale always reads 100g heavy

A broken scale always errs the same way.No matter how many times you weigh, it always reads 100g heavy.

an unbiased estimator = it may miss once, but average it over many samples and it lands on the true valuethe three readings differ, but their average points right at the true weight

An unbiased estimator is a scale whose average over many weighings lands on the true value.The three readings differ, but their average points right at the true weight.

Average over many samplesbias only shows in the average of many tries
Use an unbiased formulafor variance, divide by n-1, not n
Sample at randomdon't always pick the same box
Don't judge from just one readingit's fine if a single reading misses
A BIASED FORMULAsum of squared deviations ÷ nslightly underestimates the true spreadaverages out smaller than the truthAN UNBIASED FORMULAsum of squared deviations ÷ (n-1)corrects for one degree of freedomaverages right onto the truth

For sample variance, the divisor alone (n or n-1) can create bias.Sample variance needs n-1, not n, to stay unbiased — a degrees-of-freedom correction.

this scale is unbiased, but its readings swing a lota slightly biased but steadier scale can sometimes be more usefulunbiasedness only promises no systematic tilt — not a small spread

Being unbiased doesn't rule out a wide swing.Unbiasedness only promises no systematic tilt — not a small spread.

In one breath

An unbiased estimator may miss on any single try, but averaged over many samples, it lands on the true value.

In statistics, an estimator θ̂ is unbiased if its expected value E[θ̂] equals the true parameter θ. The sample mean is an unbiased estimator of the population mean, but the naive sample variance (dividing by n) is biased — dividing by n-1 instead (Bessel's correction) makes it unbiased. Unbiasedness only guarantees no systematic tilt, which is separate from variance (efficiency); in practice, estimators that trade a little bias for less variance are sometimes preferred — the bias-variance tradeoff.

When grown-ups say it

Estimator
The scale that guesses the truth from a sample. Each formula you use is itself an estimator.
Parameter
The true average weight of the whole truck. The true value we can never directly see.
Sample Mean
The average weight of one box. The classic unbiased estimator of the population mean.
Degrees-of-Freedom Correction
Shaving one off the number you divide by. Makes the sample variance unbiased.
Bias-Variance Tradeoff
The tug-of-war between no tilt and no swing. Having both no tilt and no swing is rarely possible.
Regression Assumptions
The conditions that keep the scale hanging straight. Unbiasedness needs some of these, such as linearity and exogeneity. → see it in full
Cross Validation
Splitting into groups and testing in rotation. Shares the idea of repeated estimation. → see it in full
Time-Series Analysis (AR/MA)
A scale that guesses tomorrow from the last few days. The scale behind a forecast is checked for bias too. → see it in full
Next story
The Four Conditions for Hanging the Scale Straight →
Want another term as a picture book? Request →