LLM RAM Calculator — Run Models on CPU
Can you run a model in system RAM (CPU or unified memory), and how fast? This is the case that opens up with large Mixture-of-Experts models — only a fraction of the parameters activate per token, so models far too big for consumer VRAM run at usable speeds in RAM. Pick your model, quant, and memory, and see whether it fits and roughly how fast it'll decode. Uses the same weight and KV cache math as theVRAM calculator.
Read the estimate as a feasibility ballpark (±30%).This is decode speed only — prompt processing (prefill) on CPU is compute-bound and much slower, so long prompts mean a longer wait before the first token. Speed also depends heavily on your inference engine (ik_llama.cpp beats mainline llama.cpp for MoE), NUMA layout, and CPU. Assumes pure CPU / unified memory; hybrid GPU+CPU offload isn't modeled yet. Running on a GPU or a Mac? Use theVRAM calculator — it handles unified-memory Macs directly.
How this works
Two separate questions. Fit: every parameter has to live in RAM, so the model's full weight size at your quant, plus the KV cache and a small overhead, must be under your installed memory — computed with the exact same math as the VRAM calculator. For an MoE, that means all 235B of a 235B-A22B model must be resident, even though only 22B run per token.
Speed: CPU decode is memory-bandwidth-bound — each token requires reading the active weights plus the KV cache from RAM. So tokens/sec ≈ memory bandwidth ÷ bytes-per-token, at about 50% real-world efficiency (calibrated against measured runs; CPUs realise less of peak bandwidth than GPUs). For MoE models the per-token read is the active parameters, which is the whole reason a 235B model is viable in RAM while a dense 70B is painfully slow — the MoE simply has less to read each token.
FAQ
How much RAM do I need to run an LLM locally?
Enough to hold the entire model at your chosen quantization, plus the KV cache and a small overhead. Unlike GPU inference, all of the model's weights must fit in system RAM even for MoE models — only a fraction is active per token, but every expert has to be resident. Pick your model, quant, and RAM above and this shows whether it fits and roughly how fast it'll generate on CPU.
Can I run a large model on CPU without a GPU?
Yes, and it's increasingly common with Mixture-of-Experts (MoE) models. Because only a small share of an MoE's parameters activate per token, models that would never fit in consumer VRAM run at usable speeds in system RAM. The binding constraints become RAM capacity (to hold all the weights) and memory bandwidth (which sets decode speed), not GPU VRAM.
Why can a 235B MoE run on CPU but a 70B dense model crawls?
Decode speed is set by how many bytes must be read from memory per token. A dense 70B reads all 70B parameters every token. An MoE like Qwen3-235B-A22B reads only its ~22B active parameters per token, even though all 235B must be resident in RAM. So the MoE has ~3x less to read per token despite being far larger overall — that's why it generates faster on the same memory bandwidth. This calculator uses active parameters for the speed estimate and total parameters for the fit check.
How accurate are these CPU speed estimates?
They're feasibility ballparks, ±30%. Decode on CPU is memory-bandwidth-bound, so speed is estimated as effective bandwidth divided by bytes-read-per-token, at roughly 50% real-world bandwidth efficiency (calibrated against measured runs, and lower than the ~70% GPUs achieve). Actual speed varies more than on GPU — the inference engine matters a lot (ik_llama.cpp's fused MoE kernels beat mainline llama.cpp), as do NUMA layout and CPU generation. Treat the number as 'is this feasible and roughly how fast,' not a precise figure.
Does this include prompt processing speed?
No — this estimates decode (token generation) only. Prompt processing (prefill) on CPU is compute-bound rather than bandwidth-bound and is dramatically slower, especially for long prompts. If you feed a long document to a model running on CPU, the wait before the first token can be much longer than the decode speed here suggests.
What about running partly on GPU and partly in system RAM?
Hybrid GPU+CPU offload (e.g. keeping attention layers on a GPU while experts live in RAM, via llama.cpp's --n-cpu-moe) is a real and effective setup, but it isn't modeled here yet — this tool assumes pure CPU or unified-memory inference. Hybrid offload is on the roadmap.