Ollama Is Using CPU Instead of GPU: The Real Cause

You installed Ollama, the model runs, and your GPU sits at 5% while the CPU fans spin up. Most guides send you to driver reinstalls. That is rarely it.

The usual cause is that the model plus its KV cache no longer fits, so llama.cpp — which Ollama runs underneath — quietly moves some layers to system RAM. It does not fail. It just gets slow.

Take a 14B model at Q4_K_M on a 12 GB card, which gives about 10.8 GB usable. Layers stay on the GPU until the cache pushes them off:

• ▸num_ctx 2048 — all 40 layers on the GPU.
• ▸num_ctx 4096 — all 40.
• ▸num_ctx 8192 — still all 40.
• ▸num_ctx 16384 — 31 of 40. Nine layers now run on the CPU.

Nothing about the model changed between those lines. Only the context did, and the cache it demands grew with it.

Run your model, then in another terminal:

ollama ps
# look at the PROCESSOR column: 100% GPU, or a CPU/GPU split

nvidia-smi
# and check nothing else is holding VRAM

If ollama ps reports a split, this is your problem and no driver reinstall will fix it.

• ▸Lower num_ctx. The single biggest lever, because the cache scales with it. Use the smallest window that holds your actual prompts.
• ▸Drop one quantisation rung. Q4_K_M to Q3_K_M frees about 1.75 GB on a 14B, which is usually enough to bring every layer back.
• ▸Enable flash attention, which reduces peak attention memory at no quality cost.
• ▸Close the browser. A compositor and a few tabs can hold a surprising amount of VRAM, and Ollama only gets what is left.
• ▸Check nothing else is resident — nvidia-smi will show another model still loaded from an earlier session.

What will not help: more system RAM. That lets a bigger model load, but speed follows the bandwidth of whichever memory holds the weights, and system RAM is roughly ten times slower than a card.

Every figure here is computed the same way the calculators do it: weights are parameters times bits-per-weight divided by eight, plus a KV-cache term for your context, plus about a gigabyte of runtime overhead. Usable memory is roughly 90% of nameplate on a discrete card.