Local LLMs on CPU only (32 GB system RAM)
usable memory ≈ 25.6 GB · system-RAM assumption: 80%
Slow but workable: ~25.6 GB usable for weights. 14B at Q8 or 32B at Q4, at browsing-speed tokens.
Model verdicts at Q4 and Q8
| Model | Q4 weights | Q8 weights |
|---|---|---|
| Llama 3.1 8B8B | Runs well4.6 GB | Runs well8.5 GB |
| Llama 3.3 70B70B | Too big40.6 GB | Too big74.2 GB |
| Mistral 7B7B | Runs well4.1 GB | Runs well7.4 GB |
| Mixtral 8x7B47B | Too big27.3 GB | Too big49.8 GB |
| Qwen2.5 7B7B | Runs well4.1 GB | Runs well7.4 GB |
| Qwen2.5 14B14B | Runs well8.1 GB | Runs well14.8 GB |
| Qwen2.5 32B32B | Runs well18.6 GB | Too big33.9 GB |
| Qwen2.5 72B72B | Too big41.8 GB | Too big76.3 GB |
| Gemma 2 9B9B | Runs well5.2 GB | Runs well9.5 GB |
| Gemma 2 27B27B | Runs well15.7 GB | Too big28.6 GB |
| Phi-4 14B14B | Runs well8.1 GB | Runs well14.8 GB |
| Muse Spark Open 816B816B | Too big473.3 GB | Too big865 GB |
How to read this
Each cell shows the estimated weight size at that quantization (parameters × bytes-per-weight) and whether it fits in this machine’s usable memory plus a 2 GB headroom for context and buffers. “Tight fit” means the weights load but long contexts will struggle — prefer Q4, shorter contexts, or a smaller model.
Speed is a separate question from capacity: a fit verdict does not promise fast generation, especially on CPU-only setups where single-digit tokens per second are normal for 7B+ models.