What is the cheapest GPU that runs this model?
Work backwards. Pick the model you want to run, and the arithmetic names the cheapest card that holds it — often a rung lower than people assume.
The short answer
Computed from this tool’s default settings — model size and the rest as most people start. Change them below for your own case.
A 8B model at Q4_K_M needs 6.89 GB, which Raspberry Pi 5 covers for $120 at roughly 3 tok/s.
Cheapest GPU for a Model
The lowest-cost card that fits, at every quantisation level.
A 8B model at Q4_K_M needs 6.89 GB, which Raspberry Pi 5 covers for $120 at roughly 3 tok/s.
| Quantisation | Needs | Cheapest card | Price |
|---|---|---|---|
| Q8_0 | 10.9 GB | Arc B580 | $249 |
| Q6_K | 8.99 GB | Raspberry Pi 5 | $120 |
| Q5_K_M | 8.09 GB | Raspberry Pi 5 | $120 |
| Q4_K_M | 6.89 GB | Raspberry Pi 5 | $120 |
| Q3_K_M | 5.89 GB | Raspberry Pi 5 | $120 |
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Model size | 8B | 6 options, from 3B to 405B. |
| Context length | 8192 tokens | Anywhere from 1,024 to 131,072 tokens. |
Real answers across model size
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Model size | Cheapest at Q4_K_M | Memory needed | Price | Usable memory |
|---|---|---|---|---|
| 3B | Raspberry Pi 5 | 3.54 GB | $80 | 4.80 GB |
| 8B | Raspberry Pi 5 | 6.89 GB | $120 | 9.60 GB |
| 14B | Arc B580 | 10.7 GB | $249 | 11.0 GB |
| 32B | RX 7900 XTX | 21.8 GB | $999 | 22.1 GB |
| 70B | MacBook Pro M4 Pro | 44.5 GB | $2,799 | 48.0 GB |
A 3B model at Q4_K_M needs 3.54 GB, which Raspberry Pi 5 covers for $80 at roughly 8 tok/s.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
For each quantisation the tool computes total memory — weights, KV cache and runtime overhead — then picks the cheapest catalogued device whose usable memory covers it.
Dropping one rung down the quantisation ladder often changes which card you need by a whole price tier, which is why the table shows every rung rather than a single recommendation.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
Target a model
Find the minimum card for a specific model.
Trading quality
See how much a lower quantisation saves in hardware.
Shopping
Turn a model choice into a purchase decision.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set model size
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
1 further setting: context length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
What is the cheapest GPU that runs this model: common questions
The questions people ask about this, answered without hedging.
What is the cheapest GPU that runs a 70B model?
At Q4_K_M a 70B needs roughly 40 GB plus cache, so a 48GB card or two 24GB cards. Below that you are into partial offload, which is slow enough to change the answer.
How much does quantisation change the card I need?
Often by a whole price tier. Moving from Q8 to Q4 nearly halves the requirement, which is regularly the difference between one card and two.
Should I buy for today’s models or tomorrow’s?
Buy memory. Model sizes have not shrunk, and extra capacity is what keeps a card useful as new releases arrive.
Is a used card a reasonable answer here?
Often the best one. Previous-generation cards with large frame buffers are the value sweet spot for inference — just test the memory under load before paying.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on hardware
Will it fit
Context
Speed
Quantisation
Runtime & errors
Cost
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.