Human-in-the-Loop (HITL)

Human-in-the-loop (HITL) is a design pattern where a human reviews, approves, or corrects an AI system's output at a specific checkpoint before it takes effect, rather than letting the system act on its own from input to consequence with no one watching.

Every AI system that acts in the world has to answer one question: which decisions get made unsupervised, and which ones wait for a person? Answer it wrong in one direction and the system is too slow to be useful, buried in approval requests for things that never needed one. Answer it wrong in the other direction and a confident mistake ships at machine speed before anyone notices. HITL is the discipline of answering that question deliberately, action by action, instead of by default.

How it works

Checkpoints take different shapes depending on where they sit. Pre-action gates hold a consequential step — sending an email, deleting a record, spending money — until a human approves it, typically only for actions flagged as irreversible or high-stakes; routine, reversible actions run unsupervised so the system stays useful. Post-action review checks a batch of completed outputs before they're published or before the next batch runs, catching systemic problems rather than every single instance. Continuous review, as in labeling or moderation queues, has humans correct outputs that then feed back into improving the system over time. In agent frameworks, pre-action gating usually appears as an explicit permission policy — allow this tool automatically, but always ask a human before that one.

Why it matters

HITL is not a tax on autonomy; it's a targeting problem. The right design gates the small slice of actions where a mistake is expensive or hard to undo, and lets everything else run without a human in the way. Most of an agent's value depends on it acting mostly unsupervised — if every action needed sign-off, you'd just be doing the work yourself, slower. The craft is in scoping the gate tightly enough to catch what matters without becoming a bottleneck on what doesn't.

A worked example

An agent makes 500 tool calls in a week. Of those, 40 (8%) are gated for human confirmation because they're irreversible — sending a message, issuing a refund, deleting a file. Reviewing those 40, humans deny 6 of them (15% of the gated calls) because the action would have been wrong given context the model missed. That's 6 prevented bad outcomes out of 500 total calls — just 1.2% of everything the agent did — for the cost of reviewing 8% of its actions. A small, targeted gate catching a real, if infrequent, failure mode.

How Miatz teaches it

Learners build the always-allow/always-ask permission split themselves in the agents track, starting from an ungated loop and deciding, action by action, which calls deserve a human checkpoint — then comparing their gate list against a reference incident where an ungated action caused real damage. Mysty, the AI shadow engineer that operates inside Miatz itself, runs strictly propose-and-approve with every action logged — learners watch a production HITL system in daily use rather than only reading about the pattern.