Embedding

An embedding is a numeric vector — often 384 to 3,072 numbers long — that encodes the meaning of a piece of text, image, or other data so that similar meanings land close together in vector space.

Where a token is how a model reads text, an embedding is how a system compares text. An embedding model takes a sentence, paragraph, or document chunk and outputs a fixed-length vector. The magic is geometric: "How do I reset my password?" and "I forgot my login credentials" produce vectors that sit near each other, even though they share almost no words.

How it works

Embedding models are neural networks trained so that texts appearing in similar contexts, or labeled as similar, get pulled together in vector space while unrelated texts get pushed apart. Similarity between two vectors is usually measured with cosine similarity — the angle between them — scored from -1 to 1, where higher means more alike.

To search a document collection, you embed every chunk once, store the vectors in a vector database, then embed each incoming query and find the nearest stored vectors. Because comparing millions of vectors exactly is slow, production systems use approximate nearest-neighbor indexes such as HNSW that trade a sliver of accuracy for orders-of-magnitude speed.

Chunking matters here: embed a whole 50-page manual as one vector and every topic blurs into mush; embed single sentences and you lose context. Most pipelines split documents into chunks of a few hundred tokens, often with overlap, before embedding — the chunk size is one of the highest-leverage dials in any retrieval system.

Why it matters

Embeddings are the foundation of semantic search, retrieval-augmented generation, recommendation systems, deduplication, clustering, and anomaly detection. Keyword search finds strings; embeddings find meaning. Any time a product answers "show me things like this" or "find the document that answers this question," embeddings are almost certainly doing the work underneath.

A worked example

Suppose a support knowledge base contains three chunks:

A: "To reset your password, click Forgot Password on the login screen."

B: "Our refund policy allows returns within 30 days."

C: "Password requirements: 12 characters minimum, one symbol."

A user asks: "I can't get into my account." Embed the query, compare against the three chunk vectors, and A scores highest (say 0.78 cosine similarity), C second (0.61), B far behind (0.12) — despite the query sharing zero keywords with chunk A. That ranking is the retrieval step that feeds a RAG system its evidence.

How Miatz teaches it

The free Miatz embeddings lab lets you type sentence pairs and watch their similarity scores move, then plot a small corpus in 2D to see clusters form. Learners follow it with the chunking lab, slicing real documents different ways and measuring how retrieval quality shifts — building the intuition that vector search is geometry, not magic.