LLM Context Length Calculator
How much context can you actually run? Pick your GPU, model, and quant, and this shows the maximum context length that fits your VRAM — plus how much further Q8 and Q4 KV cache quantization get you. Built on the same engine as the VRAM calculator, so the numbers match.
The weights alone don't fit at this quant. Try a smaller quant, or see what does fit on your GPU.
| KV cache setting | Max context that fits | vs full precision |
|---|
Upper bound — real compute buffers grow with context, so usable context is lower, and decode speed drops sharply approaching the limit. Q8 KV cache is near-lossless (under ~0.1% perplexity cost); symmetric Q4/Q4 fits the most but is where quality suffers, since the value (V) cache tolerates 4-bit worse than the key (K) cache. The full calculator has asymmetric Q4-K/Q8-V presets that recover most of that for a little more memory.
Want the full VRAM breakdown?
See weights vs KV cache vs overhead at a specific context length, with per-K/V quant control.
How this works
Model weights take a fixed share of your VRAM once you pick a quant. Whatever's left, after a small overhead reserve, holds the KV cache — and since KV cache grows linearly with context length, the max context is simply how many tokens of KV cache fit in that remaining space. This tool solves that inversely for each KV cache setting, using the same VRAM math as the main calculator, so the two never disagree.
The reason KV quantization is the biggest lever for context: it directly shrinks the per-token cost. F16 stores 2 bytes each for the key and value (4 per position); Q8 stores 1 each (2 total); Q4 stores half a byte each (1 total). Halve those bytes and you roughly double the context budget — a cheaper way to gain context than dropping to a smaller weight quant, and with less quality cost, since Q8 KV is near-lossless while a weight-quant step is not.
FAQ
How much context length can my GPU run?
It depends on three things: your VRAM, the model's weight size (set by quantization), and your KV cache setting. Weights take a fixed share of VRAM; whatever's left holds the KV cache, and context length is capped by how many tokens of KV cache fit. Pick your GPU, model, and quant above and this shows the max at full-precision, Q8, and Q4 KV cache.
Does KV cache quantization let me run longer context?
Yes, and it's usually the biggest lever. KV cache scales linearly with context, so halving its bit width roughly doubles the KV budget. Q8/Q8 is near-lossless — measured at under about 0.1% perplexity cost — and a cheaper way to gain context than dropping to a smaller weight quant, which costs more quality. Q4 fits the most, but its quality cost is asymmetric: the key (K) cache tolerates 4-bit reasonably while the value (V) cache degrades faster, so an asymmetric Q4-K/Q8-V setting usually beats symmetric Q4/Q4 at nearly the same size.
Can I run more context than a model's native length?
Fitting the KV cache in VRAM is a separate question from whether the model handles that length. Past its trained context a model needs RoPE/YaRN scaling to work at all, and even then quality degrades — often its ability to reliably use information far back in the context drops before the output starts to look wrong. Zero-shot scaling is usually passable up to roughly 4x native; beyond that, full quality needs the model to have been fine-tuned for the longer length. This tool shows the VRAM ceiling and flags the model's native length so you can see which one is actually binding.
Why does longer context need more VRAM?
Every token you keep in context is stored in the KV cache — the model's memory of the conversation so far — and it grows linearly with length. At long context the KV cache can match or exceed the model weights themselves: at 128K it can be several times the size of the weight file, which is why a model that loads fine at 4K can run out of memory at 64K.
Is KV cache quantization safe for every model?
Mostly, but not universally. Most standard GQA models tolerate Q8 KV cache with negligible quality loss and Q4 with modest, asymmetric loss (values suffer more than keys). But sensitivity is architecture-dependent — a few models degrade noticeably even at Q8, while some hybrid-attention models tolerate Q4 almost perfectly. The safe move is Q8 as a near-free default, and testing Q4 on your specific model before relying on it for anything precision-sensitive.