Will it fit on your GPU?
Everyone asks it before downloading an open model — almost nobody shows the arithmetic. Pick a size, a quantization and a context length, and read the VRAM bill line by line.
| Weights — 70B params × 0.5 bytes | 33 GB |
| KV cache — 2 × 80 layers × 1024 kv-dim × 2 B × 8,192 tokens | 2.5 GB |
| Runtime overhead (~10%) | 3.5 GB |
| Estimated VRAM needed | 39 GB |
The VRAM math is shown line by line — weights, KV cache at your exact context length, and runtime overhead — not a yes/no verdict from a black box. Change one dial and watch which line moves.
Why does context length change the memory a model needs?
Every token in the context keeps its key/value tensors cached on the GPU for attention. Double the context and the KV cache roughly doubles — at long contexts it can rival the weights themselves.
What does 4-bit quantization actually do?
It stores each weight in 4 bits instead of 16, cutting weight memory to a quarter at a small quality cost. It's the single biggest lever for fitting large models on consumer GPUs.
Are these numbers exact for a specific model?
No — they're honest planning estimates for representative open-weights architectures at each size class. Real usage varies with the runtime, attention implementation and batch size; measure before you buy hardware.
This playable is part of the registry: its tool page
Next: see what your words cost
The Tokenizer Lab shows the tokens a model actually reads — the unit every context window and API bill is counted in.