Local LLMs on RTX 4070 Ti SUPER (16 GB)
usable memory ≈ 16 GB
16 GB class: 14B models at Q8, 27–32B models at Q4 with headroom checks.
Model verdicts at Q4 and Q8
| Model | Q4 weights | Q8 weights |
|---|---|---|
| Llama 3.1 8B8B | Runs well4.6 GB | Runs well8.5 GB |
| Llama 3.3 70B70B | Too big40.6 GB | Too big74.2 GB |
| Mistral 7B7B | Runs well4.1 GB | Runs well7.4 GB |
| Mixtral 8x7B47B | Too big27.3 GB | Too big49.8 GB |
| Qwen2.5 7B7B | Runs well4.1 GB | Runs well7.4 GB |
| Qwen2.5 14B14B | Runs well8.1 GB | Tight fit14.8 GB |
| Qwen2.5 32B32B | Too big18.6 GB | Too big33.9 GB |
| Qwen2.5 72B72B | Too big41.8 GB | Too big76.3 GB |
| Gemma 2 9B9B | Runs well5.2 GB | Runs well9.5 GB |
| Gemma 2 27B27B | Tight fit15.7 GB | Too big28.6 GB |
| Phi-4 14B14B | Runs well8.1 GB | Tight fit14.8 GB |
| Muse Spark Open 816B816B | Too big473.3 GB | Too big865 GB |
How to read this
Each cell shows the estimated weight size at that quantization (parameters × bytes-per-weight) and whether it fits in this machine’s usable memory plus a 2 GB headroom for context and buffers. “Tight fit” means the weights load but long contexts will struggle — prefer Q4, shorter contexts, or a smaller model.
Speed is a separate question from capacity: a fit verdict does not promise fast generation, especially on CPU-only setups where single-digit tokens per second are normal for 7B+ models.