miatz™
Log inSit Chuun
Free concept playground · no account

System Design Studio

Build an AI pipeline, then watch traffic find its breaking point.

All 90 playgrounds Join the free cohort

System Design Studio

AvailableFree

Wire an AI request path — cache, embed, vector store, rerank, generate — then ramp traffic against it. The studio finds the one stage whose throughput ceiling gives out first and names exactly how it fails, before any real request exists.

answers repeat queries before they touch the backend
5%
turns the query into a vector
13.6%
retrieves the nearest chunks
18.8%
re-scores the shortlist with a cross-encoder
33.3%
generates the grounded answer
75%
bottleneck
HoldsHolds: 10 req/s of headroom before llm saturates at 40 req/s.1014ms path latency
The node that breaks firstnamed failure mode, not a slogan

At 30 req/s the first stage to give out is llm, your tightest ceiling at 40 req/s. It still has 10 req/s of headroom before it saturates. Push traffic past 40 and it fails as: generation 429 / unbounded cost — the model is the throughput ceiling.

Every other stage could be infinitely fast — the pipeline can still only serve 40 req/s until you raise llm’s ceiling (cache in front of it, batch it, or add replicas). That’s the whole game: capacity is set by the weakest link, not the average.

Was this playground useful?

System Design Studio

Wire up an AI request path — cache, embedding, vector store, rerank, generation — then ramp traffic against it and watch it find its breaking point. Instead of a slide that says "add caching," the studio simulates load across the pipeline and lights up the single stage whose throughput ceiling gives out first, naming the exact failure mode: a cache stampede, an embedder 429, a retrieval timeout, or unbounded generation cost. The moat is that it finds and names the breaking node live, on a pipeline you assembled yourself, before any real request exists. Runs entirely in your browser (0 uploads, works offline).

How to use it

  1. Toggle stages on or off to shape your pipeline — each node shows its throughput ceiling (req/s) and the latency it adds.
  2. Set the incoming traffic with the ramp slider, or load a preset — Steady, Spike, or Adversarial burst.
  3. Watch each node's utilisation bar fill; the first stage to cross 100% is flagged as the bottleneck and the verdict tells you whether the design holds or breaks.
  4. Open the X-ray panel to read the exact failure mode of the breaking node — then unplug or reinforce it and re-run to see the ceiling move.

What this clears up (the fundamentals)

  • Capacity is the weakest link, not the average — a pipeline can only serve as many req/s as its lowest-ceiling stage. The bottleneck is always the enabled stage with the smallest maxRps, no matter how fast everything else is.
  • The bottleneck moves when you change the design — unplug the generation node and the ceiling jumps to the next-tightest stage. Optimising anything but the bottleneck buys you nothing.
  • Failures have names, not vibes — "it'll fall over under load" becomes a specific mode: cache stampede, quota 429, retrieval timeout, unbounded cost. Naming it turns a worry into a next action.
  • Latency and throughput are different axes — the generation node can be your throughput ceiling and your biggest latency add at the same time; the studio shows both so you don't conflate them.

Where it's used

A top-of-funnel AI-engineering architecture check — the fastest way to feel why "it works in the demo" and "it survives 500 req/min" are different claims. It's a miatz build-lab concept playable: play the simulation here, then learn to build the traffic engine and the first-failure detector yourself.

FAQ

How does it decide which stage breaks first?

Every stage has a throughput ceiling (maxRps). The bottleneck is the enabled stage with the lowest ceiling; if your incoming traffic exceeds that number, the pipeline breaks there first. It's a deterministic rule — no model call, no randomness.

Why does removing a node sometimes make the pipeline stronger?

Because capacity is set by the tightest ceiling. If the stage you remove was the bottleneck, the ceiling rises to the next-lowest stage, so the same traffic now holds. The studio recomputes the bottleneck every time you change the wiring.

What do the failure modes mean?

Each node fails in a characteristic way when overrun — a cache stampede when a miss storm floods the backend, a 429 when an endpoint rate-limits, a timeout when retrieval can't answer in budget, unbounded cost when generation has no ceiling. The X-ray names the one that applies to your breaking node.

Is this a real load test against my infrastructure?

No. It's a seeded simulation with fixed per-stage ceilings — a teaching model of how load finds a bottleneck, not a benchmark of any specific deployment. There is no server, no infra, and no cost.

Is anything uploaded?

No. Your pipeline and traffic settings are simulated entirely in your browser — nothing is transmitted, stored, or logged. Turn off your Wi-Fi and it still works.

Limits

This is a seeded demo: the five stages carry hand-set throughput ceilings and latencies to teach how a bottleneck emerges, not to measure your real system. Actual ceilings depend on your models, hardware, batch sizes, and network, and real pipelines fail in messier, overlapping ways. Live inference against your own architecture — routing a real model call through your key with a cost meter — is a separate BYOK mode, not this teaching simulation. The lesson it teaches (capacity is the weakest link; name the failure before you ship it) is exactly the real practice.

Related

Part of the Demystify Playgrounds. Explore the rest from the Playgrounds home.

Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.

More playgrounds

Ninety playgrounds. Zero setup.

Every concept here is playable free, no account — and inside the program you learn to rebuild the machinery yourself.

Browse all playgrounds Join free