Ollama Ignoring num_ctx? What Is Actually Happening

You paste a long document, the model answers as though it only saw the end, and nothing errors. That is not the model being stupid — it is truncation, and it is silent by design.

Ollama has historically defaulted to a small context window. Anything longer than that gets cut from the front before the model ever sees it. No warning, no error, just a model answering a question it was only shown half of.

Per session, in the interactive prompt:

/set parameter num_ctx 8192

Permanently, in a Modelfile — this is the one that survives restarts:

FROM llama3.1:8b
PARAMETER num_ctx 8192

# then:
# ollama create mymodel -f Modelfile

And through the API, where it goes in the options object on each request:

{
  "model": "llama3.1:8b",
  "prompt": "...",
  "options": { "num_ctx": 8192 }
}

Raising num_ctx works, and raising it too far quietly costs you the GPU. On a 12 GB card running a 14B model at Q4_K_M:

• ▸num_ctx 2048 — all 40 layers on the GPU.
• ▸num_ctx 4096 — all 40.
• ▸num_ctx 8192 — all 40, with about 7.96 GB of budget left.
• ▸num_ctx 16384 — 31 of 40. The other nine move to system RAM.

So the honest advice is not 'set it as high as it goes'. It is: set it to the smallest value that holds your actual prompts. 8192 covers most chat and code work. Go higher only when you genuinely feed long documents, and expect to give up either speed or model size to do it.

ollama show mymodel --parameters
# num_ctx should be listed

ollama ps
# PROCESSOR should read 100% GPU, not a split

Every figure here is computed the same way the calculators do it: weights are parameters times bits-per-weight divided by eight, plus a KV-cache term for your context, plus about a gigabyte of runtime overhead. Usable memory is roughly 90% of nameplate on a discrete card.