What is a Context Window?
The context window, told as a story about the tray in front of the parrot — how many beans fit on it at once.
After a long talk, the parrot forgot the very first thing.The name was said clearly. But after umbrellas, weather, and the cat, the parrot no longer knows the beginning.
The parrot only sees the beans on the tray. When it's full, the front beans fall off.As the chat grows, beans keep coming — the tray stays the same size. A fallen bean is a word the parrot never heard; it didn't forget, it can't see it.
The context window is the tray in front of the parrot — how many beans fit at once.A bigger tray holds a longer story at once — but there is always a limit. So put what matters at the front, summarize the long parts, and pick only the pages you need. The parrot itself: the parrot that read a mountain of books.
The note, the samples, the chat, the librarian's pages, and the parrot's answer all go on the tray.All of it is counted in beans on one tray. A long owner note leaves less room for the chat; a long answer pushes the front beans off.
A big tray doesn't mean it sees everything. And a big tray is expensive.The parrot sometimes misses beans in the middle of the tray — it sees the front and the back best. More beans cost more and take longer. Bean prices: the parrot that counts in beans.
Context window = the tray in front of the parrot: how many beans fit at once. Note, samples, chat, librarian pages, and the answer all go on it as beans; when it's full, the front beans fall off. Bigger doesn't mean it sees everything, and bigger costs more.
The maximum number of tokens an LLM can see in a single call. The system prompt, few-shot examples, chat history, RAG-retrieved documents, and the answer being generated all count toward it. Overflow means truncating the front or summarizing and compressing; even within a long context, information in the middle can be missed (lost in the middle). Repeated prefixes can be made cheaper with prompt caching.
When grown-ups say it
- Context window
- The tray. How many beans the parrot can see at once. Trays come in sizes.
- Context length
- The tray size. How many beans fit — from thousands to hundreds of thousands.
- Chat history
- Beans of everything said so far. Goes back on the tray every turn. When it gets long, the front falls off.
- Summarization · compression
- Folding a long bean row short. Shrinks old chat into a few beans to make room.
- RAG (retrieval-augmented generation)
- Pages the librarian picked. Instead of the whole book, only the needed pages go on the tray.
- Lost in the middle
- Missing the middle beans. It sees the front and back of the tray well, but sometimes misses the middle.
- Prompt caching
- Pre-counting the beans that never change. A prefix that repeats every time, like the owner note, is counted once to save cost.
- Token
- One bean. The unit that goes on the tray. → the parrot counts words in beans