How much memory does my context window cost?
Context is not free. Every token you keep in the conversation is stored as key and value vectors on the card, and that cache keeps growing while the weights stay fixed.
The short answer
Computed from this tool’s default settings — model size and the rest as most people start. Change them below for your own case.
At 8,192 tokens the cache costs 1.39 GB, which is less than the 4.50 GB of weights. Halving the context is often cheaper than dropping a quantisation level.
Context Window Memory Calculator
See the KV cache grow as you extend context, and where it overtakes the weights.
At 8,192 tokens the cache costs 1.39 GB, which is less than the 4.50 GB of weights. Halving the context is often cheaper than dropping a quantisation level.
| Context | KV cache | Total memory |
|---|---|---|
| 1,024 | 0.17 GB | 5.67 GB |
| 4,096 | 0.70 GB | 6.20 GB |
| 8,192 | 1.39 GB | 6.89 GB |
| 16,384 | 2.78 GB | 8.28 GB |
| 32,768 | 5.56 GB | 11.1 GB |
| 65,536 | 11.1 GB | 16.6 GB |
| 131,072 | 22.2 GB | 27.7 GB |
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Model size | 8B — Llama 3.1 8B class | Capacity follows total parameters, even for mixture-of-experts models. |
| Context length | 8192 tokens | The KV cache grows linearly with this. It is the biggest lever you have. |
| Quantisation | Q4_K_M — 4.5 bits/weight | Affects the weights, not the KV cache. |
Real answers across model size
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Model size | KV cache at this context | Weights | Runtime overhead | Total |
|---|---|---|---|---|
| 1B — Llama 3.2 1B class | 0.49 GB | 0.56 GB | 1.00 GB | 2.05 GB |
| 7B — Mistral / Qwen 7B class | 1.30 GB | 3.94 GB | 1.00 GB | 6.24 GB |
| 14B — Qwen 14B class | 1.84 GB | 7.88 GB | 1.00 GB | 10.7 GB |
| 32B — Qwen 32B class | 2.78 GB | 18.0 GB | 1.00 GB | 21.8 GB |
| 120B — gpt-oss 120B class | 5.38 GB | 67.5 GB | 1.00 GB | 73.9 GB |
At 8,192 tokens the cache costs 0.49 GB, which is less than the 0.56 GB of weights. Halving the context is often cheaper than dropping a quantisation level.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
Memory for a model is three things added together: the weights, which are parameters × bits-per-weight ÷ 8; the KV cache, which grows linearly with context length; and about a gigabyte of runtime overhead for the CUDA context, activations and framework.
Capacity follows a model’s total parameter count even for mixture-of-experts designs, because the router may select any expert on the next token and all of them must stay resident. Only throughput follows the active count.
The KV cache stores key and value tensors for every token in the context, so it grows linearly with context length while the weights stay fixed. At long context on a large model it can rival or exceed the weights themselves.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
Long documents
Check a 32K or 128K context fits before you rely on it.
Debugging OOM
Find the context length that stops the crashes.
Serving
Budget cache per session before setting a concurrency limit.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set model size
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
2 further settings: context length, quantisation. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
How much memory does my context window cost: common questions
The questions people ask about this, answered without hedging.
Why does a longer context use more VRAM?
The KV cache stores one key and one value vector per token per layer. Double the tokens and you double the cache. Weights do not change, which is why a model can load fine and then run out of memory an hour into a conversation.
Can I reduce KV cache memory?
Yes, three ways: shorten the context, enable flash attention, or quantise the cache itself — llama.cpp exposes 8-bit and 4-bit KV cache, which halves or quarters it for a small quality cost on long contexts.
Does a 128K context model always use 128K of cache?
No. The cache is allocated for the context you configure and filled as the conversation grows. Setting a smaller --ctx-size than the model supports is the normal way to run a long-context model on a small card.
Is more context always better?
No. Attention quality thins across a very long window, and cost grows the whole time. Retrieving the right 4K usually beats stuffing 100K.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on context
Will it fit
Speed
Quantisation
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.