How much VRAM do you need for 7B, 14B, and 32B models?

The single most common question in local AI is "can my GPU run this model?" The answer comes down to one number: VRAM. Here's how to estimate it yourself in about thirty seconds – and a cheat sheet for the popular model sizes.

Source note: weight math below uses standard bytes-per-weight for common quants (Q4_K_M ~0.6, Q8_0 ~1.06, FP16 2.0); KV figures are estimates that vary by architecture. Lab-tested fit notes are from an RX 6800M 12 GB laptop – see lab benchmarks. For your exact model + context, use the LLM VRAM Calculator.

The quick answer

VRAM needed for the model alone (weights), plus ~1–3 GB for KV cache + overhead at 8K context:

Model size   Q4_K_M        Q8_0          FP16
──────────────────────────────────────────────
7–8B         ~4–5 GB      ~8–9 GB      ~15–16 GB
13–14B       ~9–10 GB     ~16–17 GB    ~30 GB
27–32B       ~18–21 GB    ~32–35 GB    ~60+ GB
70B          ~40–45 GB    ~75 GB       cloud only

Translate that to GPUs: an 8 GB card runs 7B Q4. A 16 GB card is the sweet spot – 14B Q4 fully, 32B with light offload. A 24 GB card runs 32B Q4 fully. That's why "16 GB" keeps showing up in our best GPU for local LLMs picks.

What actually eats VRAM

Three things, and you need to add all of them:

1. Model weights – the big one. Roughly parameters × bytes per weight. At Q4_K_M that's ~0.6 bytes/weight; at Q8_0 ~1.06; at FP16 exactly 2.
2. KV cache – grows with context length and batch size. Small at 8K, significant at 128K. For a 7B it's ~0.5 GB at 8K and ~3–4 GB at 64K+.
3. Overhead – CUDA context, engine buffers, tokenizer: usually ~0.5–1.5 GB on top.

So a 7B at Q4_K_M with 8K context ≈ 4.5 GB weights + 0.5 GB KV + ~0.5 GB overhead ≈ 5.5 GB. Comfortable on an 8 GB card, tight on a 6 GB one.

Do the 30-second math yourself

Formula: weights ≈ params(B) × bytes-per-weight, then add KV + ~1 GB overhead. Worked example – 14B at Q4_K_M, 16K context: 14 × 0.6 ≈ 8.4 GB weights, ~1 GB KV, ~1 GB overhead ≈ 10.5 GB. That is why 12 GB runs it and 16 GB runs it comfortably with headroom for longer context.

Common bytes-per-weight to memorize: Q4_K_M ~0.6, Q5_K_M ~0.7, Q6_K ~0.8, Q8_0 ~1.06, FP16/BF16 2.0. If a download page lists file size (e.g. 9.1 GB safetensors), that is your weights number – just add KV + overhead instead of estimating.

7B models: the entry level

• Q4_K_M (~4.5 GB) – runs on an 8 GB GPU (RTX 4060, RX 7600) with room for context.
• Q8_0 (~8 GB) – an 8 GB card is right at the edge; 12 GB is comfortable.
• FP16 (~16 GB) – you need a 16 GB card; at that point you'd usually rather run a 14B Q4.

13–14B models: the practical sweet spot

• Q4_K_M (~9–10 GB) – 12 GB runs it; 16 GB (RTX 4060 Ti 16 GB, RX 7600 XT) runs it with a long context.
• Q8_0 (~16 GB) – a 16 GB card runs this with a short context; 24 GB is comfortable.
• FP16 (~30 GB) – needs a 24 GB card with offload, or cloud.

14B Q4 at ~40–50 tok/s is the point where a local model stops feeling like a demo and starts feeling like a useful assistant.

27–32B models: where 24 GB earns its keep

• Q4_K_M (~20 GB + KV) – a 24 GB GPU (RTX 4090, RX 7900 XTX) runs 32B Q4 with an 8–16K context. At 32K context you're right at the edge.
• Q8_0 (~32 GB) – beyond consumer cards; needs a 32 GB card or cloud.

A 16 GB card can run a 30B Q4 with partial offload if you have 16–32 GB of system RAM – it works, but generation drops to a crawl (often 5–15 tok/s).

Context length is the hidden variable

Doubling context roughly doubles the KV cache. At 128K context, KV cache alone can be 3–6 GB – that's a whole quantization tier's worth of memory. If you're planning long documents, buy the bigger card or shorten your context.

Rule-of-thumb KV cache (q4 KV, batch 1) to add on top of weights:

Context    7–8B      14B       27–32B     35B MoE (3B active)
─────────────────────────────────────────────────────────
8K         ~0.5 GB   ~1 GB     ~2 GB      ~1 GB
32K        ~1–2 GB   ~2–3 GB   ~4–5 GB    ~2 GB
128K       ~3–4 GB   ~6–8 GB   ~10–14 GB  ~5–7 GB
262K       ~6–8 GB   ~12+ GB   ~20+ GB    ~8–12 GB (q8 KV higher)

MoE rows stay small because KV scales with active architecture, not total parameters – but q8 KV roughly doubles these numbers. Lab note: a 35B MoE at Q4/Q5 with q8 KV held 262K context on a 12 GB card with partial offload at 25–29 tok/s; a 27B dense at 3.7bpw (~15.6 GB weights alone) already overflowed 12 GB at modest context. See lab benchmarks for the exact runs.

MoE models break the simple table (in a good way)

Dense models fire every parameter per token; Mixture-of-Experts models like Ornith 35B or Tiel-Coder 35B fire only ~3B active parameters per token. VRAM still holds all 35B weights (~20 GB at Q4), but decode speed and KV behave like a 3B–7B model. That is why a 24 GB card runs a 35B MoE Q4 with long context comfortably, and why even a 12–16 GB card can run it with partial offload at usable speeds – while a 27B dense at the same quant already crawls.

Buying implication: if your VRAM budget is fixed, a 30–35B MoE with small actives gives you frontier-adjacent quality where a dense 32B would OOM. Check both weights fit and KV at your target context in the calculator before you assume “35B = needs 24 GB”.

When it still OOMs: troubleshooting checklist

1. Confirm which part overflowed. Weights too big means a smaller quant or smaller model; KV too big means shorter context, q4 KV instead of q8, or batch 1.
2. Kill parallel sequences. Two concurrent 64K sessions need roughly twice the KV of one. Test with a single sequence first.
3. Watch overhead creep. Some backends reserve 1–2 GB before the model loads. If you are within 1 GB of the limit on paper, you will OOM in practice – drop one tier.
4. Offload deliberately, not accidentally. If you must overflow, keep hot layers on GPU and spill the rest to fast system RAM (32 GB helps). Random full offload is what turns 40 tok/s into 5 tok/s.

Most “this GPU can’t run 14B” reports are really “this GPU can’t run 14B at 128K with q8 KV and two sessions”. Change one variable at a time and re-check with the calculator.

FAQ

Can I run a model with less VRAM than the weights need?

Only by offloading some layers to system RAM. It works, but every offloaded layer slows generation. More RAM helps; it doesn't replace VRAM.

Does CPU or GPU speed matter for the numbers above?

Not for the fit – VRAM is about capacity. Speed matters separately for tokens/sec.

What's the minimum usable setup?

8 GB VRAM + 16 GB system RAM gets you a decent 7B Q4 local assistant. From there, VRAM is the upgrade that matters.

Do MoE models need less VRAM than dense models?

Weights need the same (all experts live in VRAM), but KV and speed behave like the active size. A 35B MoE with 3B active feels like a 7B for speed and context cost – which is why it is the best quality-per-GB on 12–16 GB cards.

What is q8 KV and when does it matter?

KV cache quantization. q8 KV roughly doubles KV memory vs q4 but preserves long-context quality. At 8–32K you will barely notice; at 128K+ it can be the difference between fitting and OOM. If you live at long context, budget for it explicitly.

Why does my model OOM at long context when weights fit?

Because KV grew past your headroom. Halve the context, drop one quant tier on KV, or reduce batch/parallel sequences to one. The calculator shows weights + KV separately so you can see which one overflowed.

Bottom line

8 GB → 7B. 16 GB → 14B. 24 GB → 32B. Add ~1–3 GB for a long context, and never trust a VRAM claim without knowing the quantization. Check your exact model with the calculator below before you spend money on a card.

Sources

• llama.cpp – quantization formats and KV cache behavior
• VelsTech Solutions lab benchmarks – RX 6800M tested fit and offload runs
• LLM VRAM Calculator – weights + KV + fit verdict
• Best GPU for running LLMs locally (2026)
• ROCm installation guide (Ubuntu)