Same Prompt, Two Machines
Give one feature request to two coding machines and each ships a different build — both look plausible on screen. Then a hidden eval runs: a single acceptance criterion neither prompt ever named, and it alone decides which build was actually better. You read both outputs, bet which machine won, and reveal the eval to see how often the nicer-looking build quietly missed the point. It turns "the AI just knew what I meant" into a scored, visible list of exactly what it guessed. Runs entirely in your browser (0 uploads, works offline).
How to use it
- Read the feature request at the top of each round, then read both builds — Machine A and Machine B.
- Bet on the build you think the hidden eval will crown. Reading output is not the same as passing the criterion, so trust your judgement.
- Click Reveal the hidden eval to score every bet against the acceptance criterion neither prompt stated.
- Read the X-ray: each round shows the criterion, the true winner, and whether the nicer-looking build fooled you. Copy or share your accuracy.
What this clears up (the fundamentals)
- "It looks right" is not "it passes" — both builds render, both compile, both seem fine. The only thing that separates them is a criterion you never see until the eval runs. Output quality and correctness are different axes.
- The vibe prompt fails criteria it was never told about — a loose one-line request leaves acceptance criteria implicit, so the machine guesses. Some guesses land; two to four usually don't, and they don't announce themselves.
- The eval is the truth, not the spec — even a tighter build can regress (round five ships a login that leaks the token to JavaScript). Whoever passes the hidden criterion wins, regardless of which prompt looked more thorough.
- Writing the criterion down first is the fix — specification-driven development means stating acceptance criteria and encoding them as an eval before you ship, so "seems fine" becomes pass/fail you can see.
Where it's used
This is the free, no-signup teaser for the miatz build-lab module Vibe-Coding a Prototype You Can Ship — a demo-to-lab pair on specification-driven development, acceptance criteria, and eval-gated shipping. It's a miatz build-lab concept playable: bet against the hidden eval here, then learn to write the acceptance criteria and build the eval suite yourself.
FAQ
What is the "hidden eval"?
An acceptance criterion the feature request never named — the real test of whether a build did the job. Both machines are scored 0–100 against it, and the higher score wins. It stays hidden until you reveal it, which is the whole point: you have to judge the builds without seeing the answer key.
Why does the nicer-looking build often lose?
Because presentation and correctness are different things. A build can accept any file, apply any discount code, or store a session token in the wrong place and still look complete. The eval measures the one behaviour that matters, and flashier output frequently skips it.
Are these real coding agents running live?
No. The two builds and their eval scores are precomputed and baked into the page so it stays deterministic and network-zero. The lesson — that a hidden acceptance criterion, not the prettier output, decides the winner — is exactly how a real eval gate behaves.
What's specification-driven development?
Writing the acceptance criteria for a feature before you build it, then encoding them as an automated eval. Instead of hoping the model "knew what you meant," you state what "done" means up front and check every build against it — turning subjective review into a pass/fail gate.
Is anything uploaded?
No. Your bets are scored entirely in your browser — nothing is transmitted, stored, or logged. Turn off your Wi-Fi and it still works.
Limits
A teaching playground with five seeded rounds and precomputed, illustrative eval scores — not a live coding agent or a real CI suite. Real builds vary run to run, and real evals cover many criteria at once. The rounds are hand-authored to make one idea unmissable: a hidden acceptance criterion, not the nicer-looking output, decides which build was right.
Related
Part of the Demystify Playgrounds. Explore the rest from the Playgrounds home.
Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.