Local LLMs on Mac Studio M2 Max (64 GB unified)
usable memory ≈ 44.8 GB · macOS unified-memory assumption: 70%
~44.8 GB usable — the single-box path to 70B-class models at Q4.
Model verdicts at Q4 and Q8
| Model | Q4 weights | Q8 weights |
|---|---|---|
| Llama 3.1 8B8B | Runs well4.6 GB | Runs well8.5 GB |
| Llama 3.3 70B70B | Runs well40.6 GB | Too big74.2 GB |
| Mistral 7B7B | Runs well4.1 GB | Runs well7.4 GB |
| Mixtral 8x7B47B | Runs well27.3 GB | Too big49.8 GB |
| Qwen2.5 7B7B | Runs well4.1 GB | Runs well7.4 GB |
| Qwen2.5 14B14B | Runs well8.1 GB | Runs well14.8 GB |
| Qwen2.5 32B32B | Runs well18.6 GB | Runs well33.9 GB |
| Qwen2.5 72B72B | Runs well41.8 GB | Too big76.3 GB |
| Gemma 2 9B9B | Runs well5.2 GB | Runs well9.5 GB |
| Gemma 2 27B27B | Runs well15.7 GB | Runs well28.6 GB |
| Phi-4 14B14B | Runs well8.1 GB | Runs well14.8 GB |
| Muse Spark Open 816B816B | Too big473.3 GB | Too big865 GB |
How to read this
Each cell shows the estimated weight size at that quantization (parameters × bytes-per-weight) and whether it fits in this machine’s usable memory plus a 2 GB headroom for context and buffers. “Tight fit” means the weights load but long contexts will struggle — prefer Q4, shorter contexts, or a smaller model.
Speed is a separate question from capacity: a fit verdict does not promise fast generation, especially on CPU-only setups where single-digit tokens per second are normal for 7B+ models.