In-Context Learning

In-context learning is the ability of a large language model to infer and perform a task purely from what appears in its prompt — instructions, examples, or both — without any of its weights being updated, and without ever having been explicitly trained on that exact task.

This is the underlying capability that few-shot and zero-shot prompting both spend: neither technique changes a single parameter in the model. Everything that looks like "learning" a new task happens transiently, within the forward pass over that one prompt, and vanishes the moment the conversation ends. It is a strange kind of learning by ordinary standards, no gradient update, no persistence, no memory of it in the next session, and it was also, in its early demonstrations, a genuine surprise: nobody had explicitly trained these models to generalize novel input-output patterns from a handful of examples, yet scale made it emerge as a byproduct of next-token prediction pretraining.

How it works

The precise mechanism is still an active research question, but the behavior is well documented: shown a handful of consistent input-output pairs in the prompt, a sufficiently large model will infer the transformation those pairs demonstrate and apply it to a genuinely new input, including transformations invented on the spot that could not have appeared, as such, anywhere in training data. This is what distinguishes in-context learning from the model simply retrieving a memorized answer: the pattern itself, not the specific instance, is what gets generalized within the prompt.

In-context learning also degrades and improves with real, empirically observed factors: more examples generally help up to a point of diminishing returns, example order affects results, and larger models reliably show stronger in-context learning than smaller ones trained the same way, which is part of why the capability was largely a large-model story before it became a small-model one too.

Why it matters

In-context learning is what makes prompting a viable alternative to fine-tuning for an enormous range of tasks: instead of collecting a labeled dataset and updating model weights, slow, expensive, and requiring retraining every time requirements shift, a handful of examples in a prompt can steer behavior in seconds, for free, reversibly. It has real limits: it does not persist across sessions, it competes for context-window budget, and tasks requiring genuinely large amounts of task-specific knowledge still out-perform with fine-tuning or retrieval. But for the broad middle ground of teaching the model a specific format or judgment call, right now, it is usually the first and cheapest tool to reach for.

A worked example

Show a model three examples of a made-up letter-shift rule, never named as such: "cat" becomes "fdw," "dog" becomes "grj," "sun" becomes "vxq," each letter shifted three places forward in the alphabet. No instruction explains the rule; only the pattern is shown. Asked to apply the same transformation to a new word, "banana," a model with working in-context learning infers the shift from the three examples alone and produces "edqdqd," the correct shift-by-three output for a word it was never shown, using a cipher rule it was never told and never trained on. That generalization, from pattern to new instance, purely within the prompt, is in-context learning demonstrated directly.

How Miatz teaches it

Learners run exactly this kind of invented-rule exercise early in the prompting module, a transformation with no name and no explanation, only examples, specifically because inventing a rule the model could not have memorized is the only way to isolate in-context learning from simple recall, before moving on to few-shot and zero-shot prompting as its two practical applications.