Can You Actually Run This
Pick a model, a quantization level, and your GPU — the tool runs the real VRAM arithmetic live and gives you an honest fits / doesn't-fit verdict with every number shown. Weight bytes (parameters × bytes-per-parameter) plus a KV-cache estimate that grows with your context length plus runtime overhead, stacked against your card's memory. No "just download and run it" hype — a gauge fills, a verdict lands, and an estimated tokens/sec range tells you how fast it'll actually go. Runs entirely in your browser (0 uploads, works offline).
How to use it
- Pick a model and a GPU from the dropdowns, or tap a preset — Laptop · 3B · int4, 4090 · 13B · int8, or 4090 · 70B · fp16.
- Toggle the quantization (fp16 / int8 / int4) and drag the context-length slider, and watch the VRAM gauge fill live against your card's capacity marker.
- Read the verdict — fits or doesn't — plus the estimated tokens/sec, and if it overflows, the lighter quantization that would make it fit.
- Open the X-ray panel to see every term in the formula worked out, then copy or share your personalized result card.
What this clears up (the fundamentals)
- Model size on disk is not the whole VRAM bill — you pay for weights plus a KV-cache that scales with context length plus framework overhead. The context window quietly eats memory as it grows.
- Quantization is bytes per parameter — fp16 is 2 bytes, int8 is 1, int4 is 0.5. Halving the bytes roughly halves the weight footprint and lets a bigger model fit the same card, at some quality cost.
- Throughput moves with quantization too — cheaper weights push more tokens/sec, so int4 is both smaller and faster than fp16 on the same GPU. When a model doesn't fit, throughput collapses because the card thrashes.
- "Fits" is a single inequality — total required GB ≤ your VRAM. Every scary "can I run a 70B?" question reduces to arithmetic you can rerun by hand.
Where it's used
A FREE, top-of-funnel AI-engineering reliability check — the fastest way to feel why a model does or doesn't fit your hardware, sized to the exact GPU you picked. It's a miatz build-lab concept playable: play it here, then learn to build the VRAM-budgeting formula and the fuel-gauge reveal yourself in AIE-110.
FAQ
How is the VRAM requirement calculated?
Three visible terms, summed: weights = parameters × bytes-per-parameter (2 / 1 / 0.5 for fp16 / int8 / int4); KV-cache = a per-1k-token estimate that scales with context length and model size; overhead = 1.5 GB + 15% of the weights for the framework and activations. If the total is ≤ your GPU's VRAM, it fits. It's a deterministic rubric — no model call, no randomness.
Why does int4 run faster than fp16, not just smaller?
Because there are fewer bytes to move per weight, so more tokens flow per second — roughly 1.6× at int8 and 2.2× at int4 versus fp16. Smaller and faster is the whole appeal of aggressive quantization, traded against some output quality.
Are the tokens/sec numbers exact for my card?
No — they're a rough single-stream estimate anchored to each GPU, meant to show the shape of the trade-off (quant up, speed up; overflow, speed collapses). Your real throughput depends on the runtime, batch size, and driver. Treat it as a range, not a benchmark.
Is anything uploaded?
No. Your model, GPU, quantization, and context choices are computed entirely in your browser — nothing is transmitted, stored, or logged. Turn off your Wi-Fi and it still works.
Limits
A teaching estimator, not a profiler. The KV-cache term is a per-1k-token approximation rather than a layer-exact figure, overhead is a flat heuristic, and throughput is a rough anchor per card — real numbers shift with the serving runtime, batch size, activation checkpointing, and offload strategy. The point it makes is exactly right, though: fitting a model is arithmetic (weights + KV + overhead vs VRAM), and you can always rerun it.
Related
Part of the Demystify Playgrounds. Explore the rest from the Playgrounds home.
Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.