Open-Weights Garage
Pick an open-weight model — Mistral, Llama, Gemma, or Phi-class — choose a quantization level (fp16, int8, or int4), and point it at a GPU. The garage does the exact VRAM math and fills a server rack GB by GB in real time, so "this model doesn't fit" becomes a watchable overflow instead of an out-of-memory error you decode after the fact. Every model wears its license and its best use on the label, so you see fit, freedom, and fitness for purpose in one glance. Runs entirely in your browser (0 uploads, works offline).
How to use it
- Pick a model — its size in billions of parameters, license, and what it's good for are on each card.
- Choose a quantization level — fp16 (full precision), int8 (half the memory), or int4 (a quarter).
- Set your GPU VRAM with a preset (8 / 16 / 24 / 48 / 80 GB) or the slider.
- Watch the rack fill: green blocks are the model, grey is free VRAM, red blocks are the overflow past your card's edge.
- Read the verdict and headroom, open the X-ray for the block-by-block math, then copy or share the result.
What this clears up (the fundamentals)
- VRAM is just arithmetic, not a mystery — a model's weights are
params × bytes-per-parameter. A 7B model at fp16 is 7 × 2 = 14 GB of weights before a single token generates. - Quantization is the memory dial — int8 halves the bytes per parameter, int4 quarters them. That's how a model that overflows at fp16 slides comfortably onto the same card at int4.
- Overhead is real — weights aren't the whole bill. KV cache, activations, and the framework add roughly 20% on top, which is why "it just barely fits" often doesn't once you load a long context.
- License ≠ size — a model can be small and permissive (Apache-2.0, MIT) or large and community-licensed. Fitting on your card and being free to ship are two different questions; this shows both.
Where it's used
A top-of-funnel sovereignty and anti-lock-in hook — the fastest way to feel why open weights are a real option, not a slogan. It's a miatz build-lab concept playable for AI Engineering (AIE-110, Open-Weights badge): play the VRAM math here, then learn to build the rack visual and the fit engine yourself.
FAQ
How is the VRAM estimate calculated?
Weights are paramsB × bytes-per-parameter (fp16 = 2, int8 = 1, int4 = 0.5 bytes), then a fixed 20% is added for runtime overhead (KV cache, activations, framework). If that total is less than or equal to your VRAM, it fits. It's a deterministic rubric — no model call, no randomness.
Why does a model that overflows at fp16 suddenly fit at int4?
Because quantization changes the bytes per parameter. fp16 stores each weight in 2 bytes; int4 stores it in half a byte — one quarter the memory. Same model, same card, one quarter the footprint. Quality drops a little, but often not enough to notice.
Is this the exact memory my server will use?
It's a close teaching estimate, not a profiler. Real usage shifts with batch size, context length, the specific runtime (vLLM, llama.cpp, TGI), and paged-attention tricks. The 20% overhead is a sensible default, not a measurement of your exact stack.
What do the license labels mean?
Each model shows its license and a one-line read of what it permits — Apache-2.0 and MIT are permissive (commercial use, few strings); Llama's community license is free below a large user threshold with acceptable-use terms; Gemma has its own terms. Fitting on your card and being free to ship it are separate checks.
Is anything uploaded?
No. Every calculation runs in your browser — no model is called, nothing is transmitted, stored, or logged. Turn off your Wi-Fi and it still works.
Limits
A teaching estimator with a fixed model list and a flat 20% overhead — it answers "will this class of model fit?", not "profile my exact deployment." Real memory depends on batch size, context length, the serving runtime, and quantization method (GPTQ, AWQ, GGUF all differ slightly). The arithmetic it teaches — weights = params × bytes/param, plus overhead, versus VRAM — is exactly the real practice.
Related
Part of the Demystify Playgrounds. Explore the rest from the Playgrounds home.
Bookmark this page (Ctrl+D, or ⌘D on Mac) — it works offline the next time you need it.