Apple Silicon Macs – M1, M2, M3, and M4 – are quietly some of the best local-AI machines you can buy. Not because the GPU is the fastest, but because of unified memory: the same RAM is shared by CPU and GPU, which is exactly what running an LLM wants. A Mac with enough RAM can hold models that would need an expensive discrete GPU on a PC.
Here's what you need to know about running LLMs on a Mac, what will fit your machine, and how to get started.
Why Macs are good at local AI
An LLM's speed is mostly limited by how fast you can feed its weights to the compute units – that's memory bandwidth. Apple Silicon's unified memory means:
- No VRAM wall. On a PC you're capped by GPU VRAM (12–24 GB typically). On a Mac, the entire system RAM is usable for the model. A 64 GB M-series Mac runs models a 24 GB PC GPU can't.
- Efficient Metal backend. llama.cpp and Ollama have excellent Metal support, so models use the GPU automatically.
- Low power. Even a M-series laptop draws little power running inference, so it's a genuinely practical always-on AI machine.
The catch: RAM is everything, and it's soldered. Buy the RAM you'll want in three years, because you can't upgrade it later.
What fits your Mac
Model size at 4-bit (Q4_K_M) roughly matches RAM needed, plus the system needs a few GB for itself:
| Mac RAM | Comfortable model (Q4) | Use case |
|---|---|---|
| 8 GB | 3B–4B | Basic chat, light tasks |
| 16 GB | 7B–9B | Solid daily driver |
| 24 GB | 13B–14B | Serious local AI |
| 32 GB | 27B–30B | Large models, good context |
| 64 GB+ | 70B | Frontier-adjacent |
Memory bandwidth also matters for speed – the M-series Pro/Max chips have more bandwidth than the base M-series, so a "same RAM, bigger chip" Mac runs faster.
Getting started
The easiest path is Ollama, which detects Metal automatically. Install it, then:
brew install ollama ollama pull llama3.2:3b ollama run llama3.2:3b
Ollama uses your Mac's GPU out of the box. For more control, build llama.cpp with Metal and use it directly – same concept as the llama.cpp guide, just with the Metal backend.
New to local AI entirely? The getting started guide walks through the whole flow.
Performance expectations
Expect 5–20 tok/s decode on an M-series Mac with a model that fits comfortably – usable for chat, not instant. The exact number depends on your chip, RAM, model size, and quantization. Compare against real results in the benchmark database (we test Radeon, but the decode formula – bandwidth ÷ bytes per token – applies the same way).
Use the LLM VRAM Calculator to estimate whether a model fits your Mac's RAM at your chosen context and quantization.
Practical tips
- Buy RAM over storage. Storage is cheap to add externally; soldered RAM is forever.
- Close other apps when running big models – the model competes with them for the same unified memory.
- Prefer Q4_K_M to fit bigger models; see the quantization deep dive for the quality trade-off.
- Pro/Max chips run faster at the same RAM due to more memory bandwidth.
Bottom line
Apple Silicon is one of the most practical local-AI platforms. It is quiet and low-power. Thanks to unified memory, it can run models that would choke a PC GPU with half the RAM. A 16 GB Mac is a genuine daily-driver local-AI machine. A 32 GB+ one is a serious workstation. The key decision is buying enough RAM upfront.