What is Sharding?
Sharding, told as a story about ticket booths that split the guest log by number.
One booth holds the whole guest log.Finding a name means searching the whole thing.
With a million guests, finding a name is slow.Searching one giant ledger takes a long time.
Split booths by number, and each one scans a small slice.The search gets faster, one booth's worth at a time.
A split rule and a sign always go together.The number tells you exactly which booth to visit.
Split the numbers badly and everyone piles into one booth.And questions that cross every booth are slow.
Sharding = splitting the guest log by a rule (a number) so each booth only holds a small slice. A sign shows which booth to visit, and an overloaded booth gets rebalanced.
Splitting data across multiple servers (shards). The rule used to split is called a partition key. One shard getting overloaded is a hotspot, and a query spanning multiple shards (a cross-shard query) is slow.
When grown-ups say it
- Sharding
- Spreading the log across several booths. Each booth owns a slice, not the whole thing.
- Partition key
- The rule used to split. Something like a number or the first letter of a name.
- Hotspot
- Everyone piling into one booth. Happens when the split rule is picked badly.
- Rebalancing
- Re-splitting an overloaded booth. Done when a hotspot shows up.
- Consistent hashing
- A split method that minimizes reshuffling. In one line: splitting that avoids moving everything around.
- Cross-shard query
- A question that needs every booth added up. Why a question like "total guests" is slow.
- Replication
- Writing the same log in more than one place. A different problem from sharding — it's for safety, not for size. → the same guest log in two warehouses
- Horizontal scaling
- Adding more booths instead of a bigger one. Sharding is a classic way to scale horizontally.