Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation (RAG) is an architecture where a system first retrieves relevant documents from a knowledge source, then passes them to a language model as context so the answer is grounded in real evidence rather than the model's memory alone.

A raw LLM only knows what was in its training data — frozen at a cutoff date, missing everything private to your company, and prone to confident invention when asked about the gaps. RAG fixes this by turning the model from an oracle into a reader: give it the right pages, and let it answer from those pages.

How it works

A RAG pipeline has two phases.

Indexing (done once, offline): split your documents into chunks — typically a few hundred tokens each, often with overlap so sentences at the boundaries are not orphaned — embed each chunk into a vector, and store the vectors in a searchable index. Chunking strategy is decisive: too large and retrieval gets fuzzy, too small and answers lose surrounding context. Splitting on semantic boundaries like headings and paragraphs usually beats splitting on raw character counts.

Query time (every request): embed the user's question, retrieve the top-k most similar chunks (pure vector search, or hybrid search combining keywords and vectors, often followed by a reranker), then build a prompt that says, in effect: "Using only the following excerpts, answer the question. If the excerpts don't contain the answer, say so." The model generates a response grounded in the retrieved text, ideally with citations.

Why it matters

RAG is the workhorse pattern of enterprise AI. It lets a model answer from documents it never trained on, stay current without retraining, respect access controls (retrieve only what this user may see), and cite sources so humans can verify. It also dramatically cuts hallucination — the model no longer has to remember; it only has to read. Most "chat with your docs," support copilots, and internal knowledge assistants are RAG systems.

A worked example

An engineer asks an internal bot: "What's our retry policy for the payments service?" The pipeline embeds the question, retrieves three chunks — one from the payments runbook ("retries: 3 attempts, exponential backoff starting at 200ms"), one from an incident postmortem, one from an outdated wiki page. The prompt bundles the chunks with the question. The model answers: "Three attempts with exponential backoff starting at 200ms, per the payments runbook," citing the source. Without retrieval, the same model would have guessed — plausibly, confidently, and possibly wrong.

How Miatz teaches and uses it

RAG is both a lab and the institute's own architecture. The free chunking and RAG labs let you slice real documents, tune chunk sizes, and watch answer quality change. More fundamentally, every Miatz learner has a personal memory engine: everything you learn — journal entries, solved problems, review results — is indexed into your own personal RAG, and the AI tutor grounds its coaching in it. You are not just studying RAG; you are living inside one built from your own learning history.