Can I run an LLM without a GPU?
You can run a language model with no GPU at all. The limit is system memory bandwidth, which is roughly ten times lower than a card — so the question is which sizes stay usable.
The short answer
Computed from this tool’s default settings — system ram and the rest as most people start. Change them below for your own case.
24.0 GB of usable RAM at 50 GB/s runs up to 32B at about 2.2 tok/s. For interactive use a 3B or 8B model at Q4 is the sweet spot on CPU; anything larger is a batch job.
CPU-Only Inference Calculator
What CPU-only inference realistically gives you, at each model size.
24.0 GB of usable RAM at 50 GB/s runs up to 32B at about 2.2 tok/s. For interactive use a 3B or 8B model at Q4 is the sweet spot on CPU; anything larger is a batch job.
| Size | Needs | Speed | Feel |
|---|---|---|---|
| 1B | 1.81 GB | 71.1 tok/s | Comfortable |
| 3B | 3.11 GB | 23.7 tok/s | Comfortable |
| 8B | 6.20 GB | 8.9 tok/s | Slow but usable |
| 14B | 9.79 GB | 5.1 tok/s | Slow but usable |
| 32B | 20.4 GB | 2.2 tok/s | Batch only |
| 70B | 42.4 GB | Will not fit | — |
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| System RAM | 32 GB | Anywhere from 4 to 512 GB. |
| Memory bandwidth | 50 GB/s | DDR4 dual-channel ≈ 40, DDR5 ≈ 70, high-end workstation ≈ 200. |
| Quantisation | Q4_K_M — 4.5 bits/weight | 5 options, from Q8_0 — 8.5 bits/weight to Q3_K_M — 3.5 bits/weight. |
| Context length | 4096 tokens | Anywhere from 1,024 to 32,768 tokens. |
Real answers across quantisation
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Quantisation | Largest that fits | Usable RAM | Bandwidth | Expected speed |
|---|---|---|---|---|
| Q8_0 — 8.5 bits/weight | 14B | 24.0 GB | 50 GB/s | 2.7 tok/s |
| Q6_K — 6.6 bits/weight | 14B | 24.0 GB | 50 GB/s | 3.5 tok/s |
| Q5_K_M — 5.7 bits/weight | 14B | 24.0 GB | 50 GB/s | 4.0 tok/s |
| Q4_K_M — 4.5 bits/weight | 32B | 24.0 GB | 50 GB/s | 2.2 tok/s |
| Q3_K_M — 3.5 bits/weight | 32B | 24.0 GB | 50 GB/s | 2.9 tok/s |
24.0 GB of usable RAM at 50 GB/s runs up to 14B at about 2.7 tok/s. For interactive use a 3B or 8B model at Q4 is the sweet spot on CPU; anything larger is a batch job.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
CPU inference is bound by system memory bandwidth in exactly the way GPU inference is bound by VRAM bandwidth, so the same formula applies with a much smaller number. The result is usually single-digit tokens per second above 8B — fine for batch work, uncomfortable for chat.
Leave more headroom than on a GPU: the operating system and everything else you are running share this pool. The estimate reserves a quarter of RAM.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
No GPU
Find what your CPU and RAM can already do.
Servers
Check a CPU-only box before renting a GPU.
Batch work
See where slow-but-free is good enough.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set system ram
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
3 further settings: memory bandwidth, quantisation, context length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
Can I run an LLM without a GPU: common questions
The questions people ask about this, answered without hedging.
What counts as a usable reading speed?
Comfortable chat needs about 15 tokens per second, a little faster than most people read. Below 5 is workable for batch jobs but frustrating interactively.
Can I run an LLM without a GPU?
Yes. llama.cpp runs entirely on CPU, and 1B–8B models at Q4 are practical on ordinary desktop hardware. Above that it becomes a batch job rather than a conversation.
How fast is CPU inference?
Bound by memory bandwidth, so typically single-digit tokens per second for an 8B model on dual-channel DDR5. Smaller models are proportionally faster.
Do more cores help?
Up to a point, then bandwidth becomes the wall. Memory channels usually matter more than core count, which is why workstation platforms punch above consumer ones here.
What is the best model for CPU-only?
A 3B or 8B at Q4_K_M. They fit in ordinary RAM and stay responsive enough to hold a conversation with.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
Will it fit
Context
Quantisation
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.