Zero-Shot Prompting

Zero-shot prompting is asking a model to perform a task using only a natural-language instruction — no worked examples, no demonstrations — relying entirely on what it learned during pretraining to infer what a good answer looks like.

Every conversation with a chat model in its default mode is zero-shot: you ask a question, you get an answer, and no one showed the model a template first. It works at all because large models are pretrained on enough text — code, instructions, Q&A, documentation — that instruction-following and task inference come baked into the weights before you write a single example.

How it works

Zero-shot performance rides entirely on the pretraining and instruction-tuning data the model has already absorbed. A model has almost certainly seen thousands of sentiment-classification examples, translation pairs, and summarization instructions during training, so zero-shot prompts for those tasks tend to land well. A model has essentially never seen your company's specific ticket-triage taxonomy or a bespoke schema you invented yesterday, so zero-shot prompts for those tasks tend to wobble — not because the model is incapable, but because "reasonable summary" has thousands of pretraining analogues and "urgency label per our internal rubric" has zero.

The instruction itself is the only lever available, which is why zero-shot prompt quality depends heavily on specificity: naming the exact output format, the exact categories, and the exact edge-case behavior compensates, to a real but limited degree, for the missing examples.

Why it matters

Zero-shot prompting is the right default for genuinely general tasks — draft an email, explain a concept, translate a sentence — where adding examples would cost tokens without adding information the model doesn't already have. It becomes the wrong choice exactly where output needs a precise, non-obvious shape: a custom schema, a company-specific rubric, a label set with fine distinctions the model has no strong prior for. The practical workflow most teams land on is starting zero-shot, and reaching for few-shot examples only once a scored eval shows zero-shot output missing the mark in a specific, fixable way.

A worked example

Zero-shot: "Extract the person's name, company, and role from this bio, as JSON." Given a clean, conventionally structured bio, this usually works — the model has seen enormous numbers of "extract structured data from text" examples during training, so the pattern is well anchored.

Now push it: "Extract the deal stage from this sales note, as JSON, using our stages: Qualifying, Evaluating, Committed, Stalled." Zero-shot, the model must guess what distinguishes "Evaluating" from "Committed" from context alone, and a note like "They're comparing us against two competitors but loved the demo" could plausibly land in either bucket across reruns — there is no pretraining prior for your specific four-stage taxonomy. That gap is precisely what few-shot examples exist to close.

How Miatz teaches it

Learners write the zero-shot version of every prompting exercise first, score it against a fixed eval set, and only then add examples, so the decision to go few-shot is always evidence-driven rather than reflexive. The comparison makes visible exactly which tasks a model already "knows" from pretraining and which ones need to be taught in the prompt.