Will this model fit on my GPU?
The only question that matters before a 40 GB download is whether the weights, the KV cache and the runtime overhead all fit in the memory you actually have — not the memory printed on the box.
The short answer
Computed from this tool’s default settings — your hardware and the rest as most people start. Change them below for your own case.
A 8B model fits RTX 4090 at Q8_0, needing about 10.9 GB of the 22.1 GB available at 8,192 tokens of context.
GPU-to-Model Fit Checker
Pick a card and a model size; see the best quantisation that fits and how much room is left.
A 8B model fits RTX 4090 at Q8_0, needing about 10.9 GB of the 22.1 GB available at 8,192 tokens of context.
| Quantisation | Weights | Total needed | Verdict |
|---|---|---|---|
| Q8_0 | 8.50 GB | 10.9 GB | Fits, 11.2 GB spare |
| Q6_K | 6.60 GB | 8.99 GB | Fits, 13.1 GB spare |
| Q5_K_M | 5.70 GB | 8.09 GB | Fits, 14.0 GB spare |
| Q4_K_M | 4.50 GB | 6.89 GB | Fits, 15.2 GB spare |
| Q3_K_M | 3.50 GB | 5.89 GB | Fits, 16.2 GB spare |
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Your hardware | RTX 4090 · 24 GB | The machine the model runs on. Usable memory decides what fits and memory bandwidth decides how fast it runs, so this moves every figure below. |
| Model size | 8B — Llama 3.1 8B class | Capacity follows total parameters, even for mixture-of-experts models. |
| Context length | 8192 tokens | The KV cache grows linearly with this. It is the biggest lever you have. |
Real answers across model size
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Model size | Best quantisation that fits | Usable memory | Needs | Spare |
|---|---|---|---|---|
| 1B — Llama 3.2 1B class | Q8_0 | 22.1 GB | 2.55 GB | 19.5 GB |
| 7B — Mistral / Qwen 7B class | Q8_0 | 22.1 GB | 9.74 GB | 12.3 GB |
| 14B — Qwen 14B class | Q8_0 | 22.1 GB | 17.7 GB | 4.37 GB |
| 32B — Qwen 32B class | Q4_K_M | 22.1 GB | 21.8 GB | 0.30 GB |
| 120B — gpt-oss 120B class | Does not fit | 22.1 GB |
A 1B model fits RTX 4090 at Q8_0, needing about 2.55 GB of the 22.1 GB available at 8,192 tokens of context.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
Memory for a model is three things added together: the weights, which are parameters × bits-per-weight ÷ 8; the KV cache, which grows linearly with context length; and about a gigabyte of runtime overhead for the CUDA context, activations and framework.
Capacity follows a model’s total parameter count even for mixture-of-experts designs, because the router may select any expert on the next token and all of them must stay resident. Only throughput follows the active count.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
Before you download
Check a model fits before spending an hour pulling it.
Choosing a quantisation
See which rung of the ladder brings a model into range.
Planning a build
Work backwards from the models you want to the card you need.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set your hardware
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
2 further settings: model size, context length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
Will this model fit on my GPU: common questions
The questions people ask about this, answered without hedging.
Why does my 24GB card not fit a 24GB model?
Because weights are not the whole cost. You also pay for the KV cache, which grows with context length, and roughly a gigabyte of runtime overhead for the CUDA context and activation buffers. Drivers and your desktop take a slice before any of that.
What does "usable memory" mean?
The share of total memory a model can realistically allocate. On a discrete card that is around 90%; on a unified-memory Mac it is lower, because the operating system and display share the same pool and macOS caps what one GPU process may take.
Does a bigger model always beat a smaller one?
No. A 14B model at Q5 usually beats a 32B model that only fits at Q3, because heavy quantisation costs more quality than the extra parameters buy. Fit the largest model you can run at Q4_K_M or better, then stop.
Is this the same as the VRAM calculator?
Same arithmetic, different question. The VRAM calculator tells you how much a model needs; this one starts from your card and tells you yes or no.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on will it fit
Context
Speed
Quantisation
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.