Pick any combination of models and quantizations, choose a GPU, and this wizard builds a side-by-side comparison: weight size, KV cache across context lengths, total VRAM needed, expected decode speed, and a straight fits / tight / won't fit verdict. Runs entirely in your browser – nothing is uploaded.
Rough estimates. Real usage varies with the exact model build, tokenizer, CUDA/compute-graph overhead, and your inference engine. KV cache assumes FP16. Decode speed assumes you're memory-bandwidth-bound at ~85% of peak – typical for llama.cpp / vLLM with a single stream.
How this works – assumptions & formulas
This is an estimate, not a lab measurement. The numbers below match the live calculator logic.
1. Model weight size
weights = parameters × bytes_per_weight / GiB
Bytes per weight per quantization: Q2_K 0.35, Q3_K_M 0.45, Q4_K_M 0.61, Q5_K_M 0.66, Q6_K 0.8125, Q8_0 1.0625, FP16 2.0. Example: 8B params at Q4_K_M ≈ 8e9 × 0.61 / 1.07e9 ≈ 4.5 GB.
2. KV cache
kv_per_token = 2 × layers × kv_heads × head_dim × 2 bytes kv_cache = kv_per_token × context / GiB
The ×2 covers K and V; the trailing ×2 bytes assumes FP16 KV (the default for most engines). The table shows KV at 8K, 16K, 32K, 64K, 128K and 262K contexts so you can see how long context dominates memory.
3. Total VRAM
total = weights + kv_cache + (weights + kv_cache) × 0.06 + 0.25 GB
6% covers CUDA context / engine buffers, plus a fixed 0.25 GB floor.
4. Does it fit?
total ≤ vram × 0.95 → fits; up to vram × 1.05 → tight (run with reduced KV, or watch OOM); otherwise won't fit.
5. Decode tok/s
tok/s ≈ memory_bandwidth × 0.85 / (active_params × bytes_per_weight)
Decoding is memory-bound: each token reads the active weights once. MoE models use active params (only the routed experts), which is why a 30B-total / 3B-active MoE decodes like a much smaller model. Dense models use full params. ~85% utilization accounts for overhead.