Explain like I'm five · Database

What is Database Sharding & Partitioning?

Database sharding and partitioning, told as a story about splitting an overgrown warehouse into zones.

so many boxes that finding one takes foreverone warehouse alone can't handle all the boxes and requests

So many boxes that finding one takes forever.One warehouse alone can't handle all the boxes and requests.

split the warehouse by a careless rule, and the boxes pile into one zonethe splitting key has to be chosen carefully

Split the warehouse by a careless rule, and the boxes pile into one zone.The splitting key has to be chosen carefully.

zone Azone Bsharding = splitting the warehouse into several zones (servers)each zone only handles its own share of boxes

Sharding splits the warehouse into several zones (servers).Each zone only handles its own share of boxes.

Choose a shard keydecides which zone each box goes to
Split vertically or horizontallysharding by row; partitioning by row or column
Route to the right zonethe driver (request) goes straight to the right zone
Cross-zone queries slow downeach such query has to check several zones
sharding vs partitioningShardingPartitioningsplits across different warehousessplits zones inside one warehousehorizontal split, by rowhorizontal or verticalshard key choice is criticalzones fit the query pattern

Sharding splits data across servers; partitioning splits zones within one warehouse (server).Both hinge on choosing the right split key.

zone Azone BI have to search both zones — a joinlookups in one zone get faster, but questions spanning zones get slower

Lookups in one zone get faster, but questions spanning zones get slower.Joins, which combine several zones, slow down the most.

In one breath

Sharding/partitioning splits an overgrown warehouse into zones (or servers), each responsible for its own share.

Sharding horizontally splits data across multiple physical servers (shards) for scalability. Partitioning splits a table horizontally or vertically within one server/DB. Both pay a cost in performance and consistency when a join or transaction spans more than one shard or partition.

When grown-ups say it

Sharding
Splitting across several servers (warehouses). Choosing the shard key matters most.
Partitioning
Splitting zones within one warehouse. Can go horizontal or vertical.
Shard Key
The tag deciding which zone a box goes to. Pick it wrong and the boxes overload one zone.
Join Cost
The work of searching every zone. Grows as the data is split into more zones.
Sharding (Distributed Systems View)
The deeper story from the SRE field. Replication, consensus, and CAP are covered in depth in the SRE field. → sharding in distributed systems
Concurrency Control
Rules that stop simultaneous edits from colliding. Still needed inside each zone. → the no-collision rule
NoSQL Modeling
The free-shelf warehouse. Often makes sharding easier. → the free-shelf warehouse
Normalization
Writing shared info on one shelf once. Too much normalization means more joins, clashing with sharding. → the rule of writing it once
Next story
The Rule for How Much to Hide When Hands Overlap →
Want another term as a picture book? Request →