The Peek
Stop an A/B test early, then watch it turn out wrong.
running the same zero-effect test 100 times shows peeking-and-stopping calls a false winner far more often than the 5% everyone assumes — the real false-positive rate, counted live
What goes in, what comes out
The learner runs a simulated A/B test with a real, hidden effect size and can check the live p-value and stop the test at any point. A ghost trace then replays what would have happened had they run to the pre-registered sample size, showing how often peeking flips the declared winner.
watch the live results feed and decide when, or whether, to stop the test
a p-value-over-time curve, the learner's called winner, and a ghost trace of the 'ran to completion' outcome for comparison
true effect size (including zero — no real difference), traffic rate, pre-registered sample size, number of repeated simulations
Data L3 'Evals as the Data Team's Contract' crossover; Business and Marketing A/B modules
Concepts click when you open the machinery.
Three labs are already live and free — the same hands-on style this playground brings to its module.