What context length can Ollama fit on my GPU?
Ollama defaults to a short context and silently truncates anything longer, which is why long prompts appear to be ignored. Raising num_ctx fixes it — until the cache no longer fits.
The short answer
Computed from this tool’s default settings — your hardware and the rest as most people start. Change them below for your own case.
RTX 4090 holds a 8B model at Q4_K_M with about 97,280 tokens of context. Ollama's default is far lower, so set it explicitly.
Ollama Context Length Calculator
The largest num_ctx your card holds, with the command to set it.
RTX 4090 holds a 8B model at Q4_K_M with about 97,280 tokens of context. Ollama's default is far lower, so set it explicitly.
# per session /set parameter num_ctx 97280 # or in a Modelfile PARAMETER num_ctx 97280
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Your hardware | RTX 4090 · 24 GB | The machine the model runs on. Usable memory decides what fits and memory bandwidth decides how fast it runs, so this moves every figure below. |
| Model size | 8B — Llama 3.1 8B class | Capacity follows total parameters, even for mixture-of-experts models. |
| Quantisation | Q4_K_M — 4.5 bits/weight | 5 options, from Q8_0 — 8.5 bits/weight to Q3_K_M — 3.5 bits/weight. |
Real answers across model size
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Model size | Largest context that fits | Usable memory | Weights | Left for cache |
|---|---|---|---|---|
| 1B — Llama 3.2 1B class | 340,992 tokens | 22.1 GB | 0.56 GB | 20.5 GB |
| 7B — Mistral / Qwen 7B class | 107,520 tokens | 22.1 GB | 3.94 GB | 17.1 GB |
| 14B — Qwen 14B class | 58,368 tokens | 22.1 GB | 7.88 GB | 13.2 GB |
| 32B — Qwen 32B class | 8,192 tokens | 22.1 GB | 18.0 GB | 3.08 GB |
| 120B — gpt-oss 120B class | Weights alone do not fit | 22.1 GB | 67.5 GB | 0.00 GB |
RTX 4090 holds a 1B model at Q4_K_M with about 340,992 tokens of context. Ollama's default is far lower, so set it explicitly.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
Memory for a model is three things added together: the weights, which are parameters × bits-per-weight ÷ 8; the KV cache, which grows linearly with context length; and about a gigabyte of runtime overhead for the CUDA context, activations and framework.
Capacity follows a model’s total parameter count even for mixture-of-experts designs, because the router may select any expert on the next token and all of them must stay resident. Only throughput follows the active count.
Ollama defaults to a small context and silently truncates beyond it, which is why long prompts appear to lose information. Setting num_ctx explicitly is the fix, up to whatever your memory allows.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
Long prompts truncated
Find the num_ctx that holds your whole input.
Modelfile tuning
Set a value that fits your card instead of guessing.
RAG pipelines
Size context around retrieved chunks.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set your hardware
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
2 further settings: model size, quantisation. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
What context length can Ollama fit on my GPU: common questions
The questions people ask about this, answered without hedging.
What is num_ctx in Ollama?
The context window in tokens — how much of the conversation the model can see. Ollama has historically defaulted to a small value, so long prompts are quietly cut from the front rather than raising an error.
How do I change num_ctx?
Either per session with /set parameter num_ctx 8192, or permanently in a Modelfile with a PARAMETER num_ctx line, then ollama create. The API accepts it in the options object on each request.
Why did raising num_ctx slow everything down?
A larger cache pushes total memory past what the card holds, so Ollama offloads layers to the CPU. The fix is a smaller context, not more layers.
What num_ctx should I use?
The smallest that holds your actual prompts. 8192 covers most chat and code work; go higher only when you genuinely feed long documents.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on context
Will it fit
Speed
Quantisation
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.