Explain like I'm five · AI Theory

What is On-device AI?

On-device AI, told as a story about a sorter shrunk down and kept right in the shed, so it still works out in a field with no signal.

signal's weak out here — I can't send photos to the central stationno network means the central sorting service is unreachable

Out in a remote field, the signal doesn't reach well.No network means the central sorting service is unreachable.

waiting for signal, the crop goes softeven sending a photo to the center takes timewe need a small sorter that decides right here, right now

The crop spoils while waiting for signal.A small sorter that decides right here, right now, is what's needed.

THE SMALL SORTER IN THE SHEDworks offline, right awayPASSon-device AI = shrinking the sorter and keeping it right in your own shed, no network neededthe photo never leaves the shed — the answer comes right there

On-device AI means shrinking the sorter and keeping it right in your own shed.The photo never leaves the shed — the answer comes right there.

Shrink it downmade small via compression and quantization
Works offlinedoesn't stop even with no signal
The photo stays putnever sent to a server outside the shed
The sorter judges less finelyshrunk, it judges less finely than the central sorter
CENTRAL SERVICE (AIaaS)speed — depends on signaloffline — unusablephoto travel — leaves the farmpower — bigger machine, finer judgmentON-DEVICE AIspeed — no network waitoffline — still worksphoto travel — stays in the shedpower — a bit duller after shrinking

The central service and on-device AI are compared side by side.On-device gains responses with no network wait, offline use, and data protection — but gives up some judging precision.

PASScommon crops get fast, accurate callsa rare, unfamiliar crop gets less accurate callsshrunk to fit the device's storage and battery, it simply knows less than the big one

Shrinking the sorter limits how well it can judge.Shrunk to fit the device's storage and battery, it simply knows less than the big one.

In one breath

On-device AI means shrinking the model and keeping it right on your own device, running it right there, with no network needed.

On-device AI quantizes and lightens a model (e.g. via distillation) to run inference directly on a phone, laptop, or IoT device. There's no network round trip and data never leaves the device, which helps privacy — but storage, battery, and compute limits cap the model's size and performance compared to a central cloud model. It's the opposite choice from AIaaS (a cloud service): pick on-device when latency, offline use, or privacy matter most; pick cloud when top performance matters most.

When grown-ups say it

Quantization
The technique that lightens the sorter by storing its numbers with fewer digits. A core technique for shrinking a model. → see it in full
Latency
The time spent waiting between sending a request and getting the answer. On-device removes network delay, though the device still needs compute time.
Offline Inference
A judgment call made with no network at all. It finishes right inside the shed.
Model Compression · Distillation
Transferring only the big sorter's know-how into a small one. A step usually taken before putting a model on-device.
AIaaS
Subscribing to the sorter instead of buying one. The opposite choice from on-device. → see it in full
AI Data-Privacy Compliance
The paperwork cabinet protecting crop photos. With photos staying on-site, on-device can ease paperwork such as cross-border transfer checks. → see it in full
Edge Computing
A small computing post kept near the farm. On-device AI is one form of edge computing.
AI Trustworthiness
The certification board accuracy alone can't earn. A shrunk sorter is checked against this too. → see it in full
Next story
The Scale Dial With No Systematic Tilt →
Want another term as a picture book? Request →