Schema cage match
The same request fires sixteen times at two lanes: one lets the model free-form its JSON, the other enforces the schema at decode time. Watch the off lane's malformed responses — missing fields, wrong types, truncated output, hallucinated extra keys — crash a mock downstream app, while the on lane holds the contract. Hit break it to attack the enforced lane and watch it hold. Runs entirely in your browser (0 uploads, works offline).
How to use it
- Compare the two lanes: each cell is one response — green is valid, red crashed the app.
- Read the validity rate and crash count for each lane.
- Hit break it to raise adversarial pressure, or re-run for a new batch — the enforced lane stays at 100%.
What this clears up (the fundamentals)
- "Usually valid" is a production incident — if 1 in 5 responses is malformed, that's not a warning, it's a crash every fifth call. One bad field breaks the code that parses it.
- Validate at the boundary — never feed a raw model response straight into your app. Parse and validate against a schema first; reject or retry what doesn't conform.
- Constrained decoding makes bad output impossible — schema enforcement (structured outputs / grammars) constrains the model so it can only emit valid JSON, turning a runtime crash into a structural guarantee.
- Malformation has many shapes — missing required fields, wrong types, truncated JSON, and hallucinated extra keys all break a strict consumer. A single check at the boundary catches them all.
Where it's used
An AI-engineering reliability tool — the fastest way to feel why "the model returns JSON" is not the same as "the model returns valid JSON," and why structured outputs exist. It's a miatz build-lab concept playable; learn to build it yourself.
FAQ
What is schema enforcement / constrained decoding?
A mode where the model's output is constrained during generation to conform to a schema or grammar, so it can only produce valid structured data. Many providers expose it as "structured outputs" or JSON mode with a schema.
Why not just prompt the model to return JSON?
Because prompting is a request, not a guarantee. Under load, edge cases, or adversarial input, a free-forming model will occasionally emit malformed JSON — and your parser will crash on it.
What does "validate at the boundary" mean?
Treat every model response as untrusted input: parse it, validate it against your schema, and handle failures explicitly (reject, retry, or repair) before any downstream code touches it.
Is anything uploaded?
No. The responses are generated and validated (against a real schema) entirely in your browser — nothing is transmitted or stored. Turn off your Wi-Fi and it still works.
Limits
A teaching model with a fixed schema and a simulated malformation rate — real failure rates depend on the model, prompt, and load. The mechanism (validate at the boundary, enforce the schema) is exactly the real practice.
Related
Part of the Demystify Playgrounds. Explore the rest from the Playgrounds home.
Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.