What is an Embedding?
Embeddings, told as a story about coordinate tags that record where every word and sentence sits in meaning.
The librarian looks for a cat book and can't find the one labeled kitty.The letters are different — even though the meaning is exactly the same.
Search by letters and you miss same-meaning words — and catch wrong ones that merely look alike.Same-meaning words look different, and same-looking words can mean different things. The librarian can only see letters.
An embedding is a coordinate tag that records where each word or sentence sits in meaning.The parrot writes a tag for every word. Cat, kitty and meow-cat cluster together; car sits far away. Close in meaning means close in numbers. The shelf that stores these tags is the vector DB story.
You measure the distance between tags like with a ruler. Short means similar meaning.A whole sentence becomes one tag too. But tags written by a different parrot live on a different map, so you can't measure them together.
Now you can search by meaning. But a tag is only a rough position of meaning.Hot and cold are both about temperature, so their tags can sit close. So you find by tag, then the parrot reads and answers — see the page the librarian brought.
Embedding = a coordinate tag recording where each word or sentence sits in meaning. Close meanings get close tags, so you can search by meaning even when the letters differ. But it is a rough position — opposites can sit close.
Text (or images) turned into a numeric vector of hundreds to thousands of dimensions, produced by an embedding model trained so that similar meanings yield nearby vectors. Closeness is usually measured with cosine similarity, and only vectors from the same model are comparable. It underpins semantic search, vector DBs and RAG.
When grown-ups say it
- Embedding
- The coordinate tag. A word or sentence's meaning written as a numeric position.
- Vector
- The row of numbers on the tag. Several numbers in a row like (3, 8) — real ones have hundreds.
- Dimension
- How many numbers a tag holds. The picture is 2-D; real tags are 384, 768 or 1536 dimensions.
- Cosine similarity
- The closeness score between tags. Near 1 means same meaning, 0 means unrelated.
- Embedding model
- The tag-writing parrot. A dedicated model that makes tags. Change the parrot and every tag must be rewritten.
- Semantic search
- Searching by meaning. Search by tag distance rather than letters — kitty finds cat.
- Vector DB
- The shelf organized by tags. Where tags are shelved for fast lookup. → the shelf organized by tags
- RAG
- The page the librarian brought. The page found by tag goes to the parrot. → the page the librarian brought