Free · No signup · Transparent math
Can your GPU run local LLMs?
Pick a model and your hardware — get an honest verdict based on a published formula, not marketing numbers. Llama, Mistral, Qwen, Gemma and more, checked against RTX cards, Macs and CPU-only setups.
Llama 3.1 8B on RTX 4090 (24 GB) at Q4: Runs well
Weights need ~4.6 GB; this setup offers ~24 GB usable. Comfortable fit — weights plus context headroom. Go build something.
How the math works
1. Weight size
A model’s weights take roughly parameters × bytes-per-weight of memory: Q4 quantization ≈ 0.58 bytes per parameter, Q8 ≈ 1.06. An 8B model needs ~4.6 GB at Q4.
2. Usable memory
Discrete GPUs offer their full VRAM. Apple unified memory realistically yields ~70% for weights; CPU-only setups get ~80% of system RAM after the OS takes its share.
3. Verdict rule
Usable ≥ weights + 2 GB → runs well (the 2 GB covers context and buffers). Between the two → tight fit. Below → too big.
Popular models
All 12 modelsLlama 3.1 8B
Meta · 8B params
Meta’s workhorse 8B model. The default first local LLM — strong general performance at a size almost any modern GPU can host.
Llama 3.3 70B
Meta · 70B params
Meta’s flagship open model. Near-frontier quality, but needs roughly 40 GB at Q4 — multi-GPU, a 24 GB card at aggressive quantization, or a big-memory Mac.
Mistral 7B
Mistral AI · 7B params
The model that made 7B respectable. Compact, fast, and a long-standing favorite for CPU-only and 8 GB GPU setups.
Mixtral 8x7B
Mistral AI · 47B params
A sparse mixture-of-experts model: 47B total parameters, ~13B active per token. Fast for its quality — but all 47B of weights still must fit in memory.
Qwen2.5 7B
Alibaba · 7B params
Alibaba’s 7B, strong on coding and multilingual tasks. A top pick in the 8 GB GPU class.
Qwen2.5 14B
Alibaba · 14B params
The 14B tier of Qwen2.5 — a clear quality step from 7B that still fits a 12 GB card at Q4.
Frequently asked questions
How accurate are these VRAM estimates?
The checker uses a transparent formula: parameter count × bytes-per-weight for the quantization level, plus a 2 GB headroom rule for context and buffers. It is a capacity planner, not a benchmark — real-world usage varies with context length, batch size and inference engine. Treat "Runs well" as a green light to try, not a guarantee.
What is quantization — Q4 vs Q8?
Quantization stores each model weight in fewer bits: Q4 uses roughly 0.58 bytes per parameter, Q8 about 1.06. Q4 halves memory versus Q8 with a small quality cost, and it is the standard way to fit larger models on consumer hardware.
Why can’t my 8 GB Mac run an 8B model if the weights are ~5 GB?
macOS shares unified memory between CPU and GPU and reserves a significant portion for the system. This checker assumes about 70% of Apple unified memory is realistically available for model weights — the rest is claimed by the OS, apps and display.
Does a mixture-of-experts (MoE) model need less memory?
No. MoE models like Mixtral 8x7B activate only a fraction of parameters per token, which makes them fast — but all weights must sit in memory, so capacity requirements follow the total parameter count.
What about context length?
Longer contexts consume extra memory on top of the weights — that is exactly what the +2 GB headroom rule in the verdict formula covers. If you regularly run 32k+ token contexts, add more headroom than the tool assumes.
Is CPU-only inference worth it?
It works — the checker includes CPU setups with realistic usable-RAM assumptions — but expect single-digit tokens per second on 7–14B models. It is fine for tinkering and batch jobs, painful for chat. A used 12 GB GPU transforms the experience.