What Can I Run?
Pick your GPU and see the best local LLM it can actually run — the highest-quality quant that fits your VRAM at your chosen context length, with an estimated decode speed. Built on the same engine as the VRAM calculator, so the numbers match.
Try a shorter context length, or switch KV cache to Q8 or Q4 to free up room — smaller KV cache often makes the difference on tightly-sized cards.
Recommended
Fits at Q4_K_M-class quality or better, best model first.
| Model | Best quant | VRAM used | Headroom | Est. speed |
|---|
Fits only at reduced quality
These fit only below Q4_K_M (Q3/Q2 class). They run, but with a noticeable quality drop — worth it only when the larger model genuinely matters.
| Model | Best quant | VRAM used | Headroom | Est. speed |
|---|
Want to check a specific combination?
Pick an exact model, quant, KV split, and context length — including I-quants — in the full calculator.
How this works
For every model in the dataset, this tool walks the standard K-quant ladder from Q8_0 down and picks the highest-quality quant whose total VRAM — weights plus KV cache plus overhead — fits your card at the chosen context. It calls the exact same estimation functions as the VRAM calculator, so the two tools can't disagree.
"Best" is ranked by parameter count among models that fit at Q4_K_M-class quality or better, on the general principle that a larger model at a solid quant tends to beat a smaller one at higher precision. That's a proxy, not a benchmark — a strong 14B can outperform a weak 32B on your specific task, so use the ranking as a shortlist rather than a ruling. Speed is a bandwidth-based estimate at ~70% efficiency; treat it as ±20%.
The tool sticks to K-quants because they're the standard downloads and form a clean quality ladder. If you're trying to squeeze a larger model into tight VRAM with an I-quant, that's a deliberate power-user tradeoff — do it in the full calculator, where every quant and KV cache option is available.
FAQ
How do you decide the "best" model?
It's the largest model (by parameter count) that fits your VRAM at Q4_K_M-class quality or better, at your chosen context length. Parameter count is a capability proxy, not a benchmark — a strong 14B can beat a weak 32B on a given task — so treat the ranking as a starting point, not a verdict. Models that only fit below Q4 are shown separately as reduced-quality options rather than hidden.
Why does changing context length change the recommendation?
KV cache grows with context length and competes with model weights for the same VRAM. On a tightly-sized card the best model can flip purely on this: a 24B at Q4 might fit at 4K context but not at 8K, at which point a 14B at higher quant becomes the better pick. That's why context is a first-class input here, not an afterthought.
Are these speeds real?
They're bandwidth-based estimates: decode is memory-bandwidth-bound at batch size 1, so tokens/sec ≈ effective bandwidth ÷ bytes read per token, at roughly 70% real-world efficiency. Treat them as ±20%. Actual speed varies with backend (llama.cpp, Ollama, MLX), context length, and thermal limits.
My GPU isn't listed, or I want to check a specific model.
Choose "Custom" and enter your VRAM and memory bandwidth directly. For a specific model, quant, and KV cache split — including I-quants and per-K/V quantization — use the VRAM calculator, which this tool is built on top of.