What is Split-Brain?
Split-brain, told as a story about two control rooms, cut off from each other, each insisting it's the one in charge.
The line is down, and both claim to be on call.Different orders go out from both sides at once.
When the line breaks, each side assumes the other is gone.With no way to check, each side just decides on its own.
Both believe they're the only one on call, and give orders at once — the park falls out of sync.This is split-brain.
Only recognizing a majority means no more double claims.Keep an odd number, and have both sides pause when cut off.
Orders already sent are hard to undo.So prevention is the real fix.
Split-brain = two disconnected control rooms, each believing it alone is in charge, giving conflicting orders at the same time. Prevented by keeping an odd number of rooms and recognizing only a majority.
A state where two parts of a distributed system each believe they're the leader, caused by a network partition. Requiring a quorum (a majority) prevents it, since neither side alone can reach one and both pause. Forcibly shutting down a fake leader is called fencing.
When grown-ups say it
- Split-brain
- Two control rooms each insisting they're the real one. Conflicting orders throw the park out of sync.
- Network partition
- The line between control rooms going down. This is what causes split-brain.
- Quorum
- The majority a side must clear to be recognized as on call. Fail to reach it, and a side pauses itself. → control rooms raising hands to decide
- Fencing
- Forcibly shutting a fake leader down. Done during cleanup.
- Odd node count
- Keeping control rooms at an odd number, like 3 or 5. It keeps the majority from ever being ambiguous.
- Safe mode
- Pausing itself when it can't reach a majority. It stops before conflicting orders go out.
- Leader election
- The vote that re-picks a leader after recovery. → control rooms raising hands to decide
- Multi-region
- The same risk applies when control rooms sit in different cities. A design meant to survive losing a whole city.