Context Engineering

Context engineering is the discipline of deciding what information populates a model's context window for a given request — which system instructions, retrieved documents, tool outputs, and conversation history make the cut, in what order, and in what form — as distinct from prompt engineering's focus on how instructions are worded.

Prompt engineering asks how to phrase an instruction. Context engineering asks a different question: out of everything that could theoretically inform this answer, what actually gets included, given that the context window is finite, every included token competes for the model's attention, and irrelevant material does not sit passively — it measurably degrades performance on the material that matters.

How it works

A production context is assembled, not written — pulled together at request time from several sources: a system prompt, results from a retrieval step, relevant memory or prior session data, outputs from tool calls, and some window of recent conversation. Context engineering is the set of decisions governing that assembly: retrieval instead of dumping entire documents, because the model reads what is relevant and not what is exhaustive; summarizing or compressing older conversation turns instead of keeping full transcripts forever, because a growing history eventually crowds out everything else; and placing the most important material near the start or end of the context rather than in the middle, because that is where models are demonstrably most reliable at using what they are given.

Token budget is the constraint that forces every one of these decisions to actually be made rather than deferred. Without a budget, the default failure mode is simply including everything, which, past a certain point, produces worse answers than including less, because attention is a limited resource the model spends across whatever is in front of it.

Why it matters

As context windows have grown into the hundreds of thousands of tokens, the temptation is to treat context engineering as solved by capacity — just put everything in, there is room. Long-context evaluation results consistently show this is false: relevant information competing against a pile of irrelevant material degrades answer quality even when everything technically fits. Deciding what belongs in the window, not just whether it fits, remains the actual engineering problem, and it is the discipline that determines whether a large context window is used well or merely used fully.

A worked example

A support agent has a 128,000-token window and a 40-page product manual worth roughly 28,000 tokens. The naive approach pastes the entire manual into every request, plus the full, ever-growing chat history — by turn 50 of a conversation, history alone can run past 40,000 tokens, and the system prompt with its careful escalation rules is now a small fraction of a much larger, noisier context.

The context-engineered version retrieves only the two or three manual sections relevant to the current question, roughly 2,000 tokens instead of 28,000, and replaces raw history with a running summary capped around 1,500 tokens, refreshed each turn. Total context drops from tens of thousands of mostly irrelevant tokens to a few thousand tightly relevant ones, freeing well over 30,000 tokens of budget while measurably improving answer accuracy, because the model is no longer searching for the needle in a haystack it did not need.

How Miatz teaches it

Context engineering is drilled directly after the context-window budgeting exercise: learners take the same task and rebuild its context three ways — dump everything, retrieve selectively, retrieve plus summarize — and score all three against a fixed eval set, so the lesson that more context and better context are different things is measured, not asserted.