Local LLM hardware database
Each hardware page lists which models fit — and at which quantization. Usable memory accounts for macOS reserving a share of unified memory and OS overhead on system RAM.
NVIDIA GPUs
RTX 4090 (24 GB)
usable ≈ 24 GB
The consumer king for local LLMs: 24 GB fits Qwen2.5 32B / Gemma 2 27B at Q4 outright.
RTX 3090 (24 GB)
usable ≈ 24 GB
Last gen’s 24 GB value king. Used-market favorite for exactly this workload.
RTX 4070 Ti SUPER (16 GB)
usable ≈ 16 GB
16 GB class: 14B models at Q8, 27–32B models at Q4 with headroom checks.
RTX 4060 Ti (16 GB)
usable ≈ 16 GB
The budget 16 GB option — slower than a Ti SUPER but same memory capacity class.
RTX 3060 (12 GB)
usable ≈ 12 GB
The classic entry point for local LLMs: 8–14B models at Q4 fit with room for context.
RTX 4060 (8 GB)
usable ≈ 8 GB
8 GB is the practical floor for GPU acceleration — 7–9B models at Q4, tighter at Q8.
Apple Silicon (unified memory)
MacBook Air M2 (8 GB unified)
usable ≈ 5.6 GB
Unified memory, but macOS keeps a share: ~5.6 GB realistically usable for weights.
MacBook Pro M4 Pro (48 GB unified)
usable ≈ 33.6 GB
~33.6 GB usable — 32B models at Q8 or 70B-class at Q4 are in range.
Mac Studio M2 Max (64 GB unified)
usable ≈ 44.8 GB
~44.8 GB usable — the single-box path to 70B-class models at Q4.