Context Window Lab
Hide a single fact somewhere inside a long context, then watch how likely the model is to find it again. Drag three dials — context length, needle depth, and distractors — and the lab renders a retrieval probability plus a live degradation curve across every depth. The moat is what it shows: recall is high at the edges of the window and collapses in the middle ("lost in the middle"), and it keeps thinning as you stuff more in. Instead of citing someone else's benchmark chart, you generate the curve on your own setup. Runs entirely in your browser (0 uploads, works offline).
How to use it
- Pick a context length (1–200K tokens) — the size of the haystack the fact is buried in.
- Set the needle depth: 0 = the very start, 50 = the dead middle, 100 = the very end.
- Add distractors — look-alike blocks that compete for the model's attention.
- Read the retrieval gauge (found or lost) and open the X-ray to see the U-shaped curve. Load a preset — Edge, short, Typical chat, or Buried, long — to feel the extremes.
What this clears up (the fundamentals)
- A bigger window is not a bigger memory — attention does not spread evenly. Models attend most to the start and end of a context and least to the middle, so a fact's position decides whether it survives.
- "Lost in the middle" is a shape, not a bug — it is how attention is distributed, not something you can prompt away. Put the fact that matters at an edge, not buried halfway down.
- Context rot is real — as the window fills, retrieval degrades everywhere. A fact that would be used in a lean prompt gets ignored in a stuffed one, even at the same depth.
- Distractors cost you — every look-alike block you add dilutes attention on the one that matters. More context is not free; over-stuffing is a retrieval tax.
Where it's used
A top-of-funnel AI-engineering reliability check — the fastest way to feel why "my long AI chat got dumb" is a measurable curve, not a mood. It's a miatz build-lab concept playable: play the needle-in-a-haystack here, then learn to build the retrieval model and the degradation curve yourself in AIE-103.
FAQ
What is "needle in a haystack" testing?
You place one specific fact (the needle) inside a large body of filler text (the haystack), then ask a question that requires that fact. Rerunning it at different context sizes and depths maps exactly where the model stops being able to retrieve it.
Why does the middle score worst?
Long-context attention is U-shaped: it is strongest at the beginning and end of the window and weakest in the middle. That's the "lost in the middle" effect — a fact at 50% depth is the hardest to recall, which is why the curve dips there.
Does a longer context always hurt?
At a fixed depth, yes — this lab models context rot, where retrieval thins as the window grows. In real systems a longer window can still help if it lets you place the right fact at an edge; the lesson is to spend context deliberately, not to max it out.
How is the probability calculated?
It's a deterministic teaching formula: a U-shaped depth factor (high at the edges, lowest at the middle) multiplied by a length penalty and a distractor penalty. No model is called, nothing is random — the same dials always give the same curve.
Is anything uploaded?
No. Every dial is scored entirely in your browser — nothing is transmitted, stored, or logged. Turn off your Wi-Fi and it still works.
Limits
This is a teaching model of a real effect, not a benchmark of any specific model. Actual "lost in the middle" curves vary by architecture, training, and prompt, and some models handle long context far better than others. Use the lab to build intuition for why position and size matter — then measure your real stack with your own needle-in-a-haystack runs.
Related
Part of the Demystify Playgrounds. Explore the rest from the Playgrounds home.
Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.