What is Memory?
LLM memory, told as a story about the parrot's notebook — kept outside the chat and placed on the tray first next time.
Yesterday the guest said I hate coffee. Today the parrot recommends coffee.It was said clearly yesterday. Why doesn't the parrot know?
The tray is emptied when the chat ends. The parrot has no memory of its own.The parrot only sees what is on the tray. A new chat starts with an empty tray, so yesterday might as well not have happened.
Memory is the parrot's notebook, kept outside the chat. Next time, it goes on the tray first.The parrot isn't remembering. The notebook is placed on the tray, so it looks like it remembers. When the notebook grows long, it is searched by tag like a shelf.
There are two kinds. This chat's tray is short-term; the notebook is long-term.The tray holds everything said right now; the notebook only what is needed next time. When the notebook gets very long, only the relevant lines are pulled out, like from the shelf organized by tags.
A wrong line in the notebook stays wrong every time. And the notebook is personal data, so handle it with care.The guest must be able to see, fix and erase the notebook. Keeping personal data safe is the security world's privacy story. Knowledge like company rules belongs not in the notebook but in the page the librarian brought.
Memory = the parrot's notebook kept outside the chat. The parrot has no memory and the tray is emptied when the chat ends, so what matters is written down and placed on the tray first next time. A wrong line stays wrong, and it is personal data.
An LLM is stateless: the context window vanishes when the session ends. Memory keeps conversation history, a user profile or summaries in a store outside the model and injects them into the next session's prompt. Short-term memory is the current context; long-term memory is external storage (summary, key-value or vector memory) — and because it is personal data, users need to view, correct and delete it.
When grown-ups say it
- Memory (long / short-term)
- Notebook / tray. Short-term is this chat's tray; long-term is the notebook outside the chat.
- Conversation history
- Everything said so far. Goes on the tray as is. When it grows, the earliest part falls off.
- User profile
- The guest section of the notebook. Name, likes, preferred tone and such.
- Summary memory
- A long chat in a few lines. The tray is small, so past chats are summarized into the notebook.
- Vector memory
- A notebook searched by tag. When the notebook is long, only relevant lines are pulled, shelf-style. → the shelf organized by tags
- Session
- One sitting of conversation. A fresh tray comes out when it starts and is emptied when it ends.
- Personal data
- The guest's story in the notebook. The guest must be able to view, correct and delete it. → privacy (security)
- Context window
- The tray. Where the notebook goes. It has a fixed size. → the tray in front of the parrot