What is the best model my hardware can run?
The best model is not the biggest one that loads — it is the biggest one that loads at a quantisation worth using and still generates faster than you read.
The short answer
Computed from this tool’s default settings — your hardware and the rest as most people start. Change them below for your own case.
RTX 4090 runs up to a 32B model at Q4_K_M. Anything larger only fits at a quantisation that degrades noticeably.
Best Model for My Hardware
The largest model that fits at a quantisation worth running.
RTX 4090 runs up to a 32B model at Q4_K_M. Anything larger only fits at a quantisation that degrades noticeably.
| Model size | Best quant | Memory | Speed |
|---|---|---|---|
| 1B | Q8_0 | 2.55 GB | 759 tok/s |
| 3B | Q8_0 | 5.04 GB | 253 tok/s |
| 7B | Q8_0 | 9.74 GB | 108 tok/s |
| 8B | Q8_0 | 10.9 GB | 95 tok/s |
| 14B | Q8_0 | 17.7 GB | 54 tok/s |
| 27B | Q4_K_M | 18.7 GB | 53 tok/s |
| 32B | Q4_K_M | 21.8 GB | 45 tok/s |
| 70B | — | — | Does not fit |
| 120B | — | — | Does not fit |
| 405B | — | — | Does not fit |
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Your hardware | RTX 4090 · 24 GB | The machine the model runs on. Usable memory decides what fits and memory bandwidth decides how fast it runs, so this moves every figure below. |
| Context length | 8192 tokens | The KV cache grows linearly with this. It is the biggest lever you have. |
Real answers across your hardware
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Your hardware | Largest worth running | Usable memory | Bandwidth | Expected speed |
|---|---|---|---|---|
| NVIDIA B200 · 180 GB | 120B at Q8_0 | 165.6 GB | 7,700 GB/s | 91 tok/s |
| Cerebras WSE-3 · 44 GB on-chip SRAM | 32B at Q8_0 | 44.0 GB | 21,000,000 GB/s | 933,333 tok/s |
| Raspberry Pi 5 · 16 GB | 8B at Q6_K | 9.60 GB | 17 GB/s | 3 tok/s |
| RTX 4080 SUPER · 16 GB | 14B at Q6_K | 14.7 GB | 736 GB/s | 75 tok/s |
| RTX 5070 · 12 GB | 14B at Q4_K_M | 11.0 GB | 672 GB/s | 68 tok/s |
NVIDIA B200 runs up to a 120B model at Q8_0. Anything larger only fits at a quantisation that degrades noticeably.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
Memory for a model is three things added together: the weights, which are parameters × bits-per-weight ÷ 8; the KV cache, which grows linearly with context length; and about a gigabyte of runtime overhead for the CUDA context, activations and framework.
Capacity follows a model’s total parameter count even for mixture-of-experts designs, because the router may select any expert on the next token and all of them must stay resident. Only throughput follows the active count.
Models that only fit at Q3_K_M are excluded from the recommendation. The degradation there is visible, and a smaller model at Q4_K_M is usually the better choice than a larger one at Q3.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
Getting started
A shortlist for your specific card.
After an upgrade
Find what the new hardware unlocks.
Picking a daily driver
Balance capability against responsiveness.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set your hardware
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
1 further setting: context length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
What is the best model my hardware can run: common questions
The questions people ask about this, answered without hedging.
What is the best local LLM for my GPU?
Whatever fits at Q4_K_M or better with room for your context, and still generates at reading speed. On 8GB that is a good 7–8B model; on 24GB a 32B; on 48GB or more a 70B.
Should I run the biggest model that fits?
Only if it fits at Q4 or higher. A 32B squeezed into Q3 is usually worse than a 14B at Q5, and it will be slower as well.
Does the best model depend on the task?
Yes. Coding models beat general ones at code even when smaller, and a 7B tuned for your domain can beat a 70B generalist. Size is the constraint, not the goal.
How often does the answer change?
Frequently — a new release can move the ceiling for a given card. Check again after any major model launch rather than assuming last year’s pick still holds.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on will it fit
Context
Speed
Quantisation
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.