What are a Data Warehouse and a Data Lake?
A data warehouse and a data lake, told as a story comparing a tidily shelved warehouse with a yard where things just get piled up.
Neat boxes and messy piles arrive all at once.Each item has to be either shelved now or piled up for later.
Shelving an item requires every sorting rule to be set in advance.A brand-new kind of item has no rule, so it can't go on the shelf.
A warehouse sorts things onto shelves; a lake piles them up exactly as they arrived.Routine reports use a warehouse and raw storage uses a lake, so you often need both.
The key difference between a warehouse and a lake is when data gets sorted.A warehouse sorts when data is stored; a lake sorts when data is read.
An unmanaged lake turns into a 'data swamp'.Analysis (OLAP) usually runs on the tidy warehouse side.
A data warehouse is a tidy, pre-sorted warehouse; a data lake is a raw storage yard that keeps things exactly as they arrived.
A data warehouse fixes its schema before storing (schema-on-write), holding structured data well-suited to fast aggregation and reporting. A data lake stores first and interprets structure only when needed (schema-on-read), so it can keep anything — structured or not — exactly as it arrived. Left unmanaged, a lake easily becomes a 'data swamp'; a lakehouse approach tries to combine both.
When grown-ups say it
- Schema-on-write
- Sorting before anything goes in. The approach a warehouse uses.
- Schema-on-read
- Storing first, sorting when it's used. The approach a lake uses.
- Structured/Unstructured Data
- Neat boxes vs. messy piles. A lake takes in both kinds.
- ETL/ELT
- A difference in the order of moving data and tidying it. ETL tidies before the move, ELT after.
- Data Swamp
- A yard nobody's kept up. Becomes a pile nobody can search.
- BI Reporting
- The routine aggregate report. What a warehouse is best suited for.
- Lakehouse
- A yard with a few shelves added in. An approach combining warehouse and lake.
- OLAP
- Slicing all the data by several axes for analysis. Usually run on warehouse data. → the manager who slices the whole warehouse