How much VRAM does a model of this size need?
Parameters are not gigabytes until you pick a precision. The conversion is one line of arithmetic, and it is the line that decides whether a model is downloadable.
The short answer
Computed from this tool’s default settings — parameters and the rest as most people start. Change them below for your own case.
A 8.0B model needs about 6.89 GB at Q4_K_M with 8,192 tokens of context. A bigger model at Q4 generally beats a smaller one at Q8, so spend spare memory on parameters before precision.
Model Size to RAM/VRAM Converter
Convert a parameter count into real memory at every quantisation.
A 8.0B model needs about 6.89 GB at Q4_K_M with 8,192 tokens of context. A bigger model at Q4 generally beats a smaller one at Q8, so spend spare memory on parameters before precision.
| Quantisation | Weights | Total | Trade-off |
|---|---|---|---|
| Q8_0 | 8.50 GB | 10.9 GB | Near-lossless. Use when memory is free. |
| Q6_K | 6.60 GB | 8.99 GB | Very close to FP16; the safe default if it fits. |
| Q5_K_M | 5.70 GB | 8.09 GB | Mild loss, noticeably smaller. |
| Q4_K_M | 4.50 GB | 6.89 GB | The sweet spot most people run. |
| Q3_K_M | 3.50 GB | 5.89 GB | Visible degradation. Fallback only. |
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Parameters | 8 billions | Any value — this is not limited to catalogued models. |
| Context length | 8192 tokens | The KV cache grows linearly with this. It is the biggest lever you have. |
Real answers across context length
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Context length | Needed at Q4_K_M | Weights at Q4_K_M | KV cache | Overhead |
|---|---|---|---|---|
| 1,024 | 5.67 GB | 4.50 GB | 0.17 GB | 1.00 GB |
| 33,536 | 11.2 GB | 4.50 GB | 5.69 GB | 1.00 GB |
| 66,048 | 16.7 GB | 4.50 GB | 11.2 GB | 1.00 GB |
| 98,560 | 22.2 GB | 4.50 GB | 16.7 GB | 1.00 GB |
| 131,072 | 27.7 GB | 4.50 GB | 22.2 GB | 1.00 GB |
A 8.0B model needs about 5.67 GB at Q4_K_M with 1,024 tokens of context. A bigger model at Q4 generally beats a smaller one at Q8, so spend spare memory on parameters before precision.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
Memory for a model is three things added together: the weights, which are parameters × bits-per-weight ÷ 8; the KV cache, which grows linearly with context length; and about a gigabyte of runtime overhead for the CUDA context, activations and framework.
Capacity follows a model’s total parameter count even for mixture-of-experts designs, because the router may select any expert on the next token and all of them must stay resident. Only throughput follows the active count.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
Reading a model card
Turn "70B" into a number your hardware understands.
Comparing quants
See the file size at every rung at once.
Disk planning
Work out storage before pulling a family of models.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set parameters
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
1 further setting: context length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
How much VRAM does a model of this size need: common questions
The questions people ask about this, answered without hedging.
How do I convert parameters to GB?
Multiply parameters by bits-per-weight and divide by eight. A 7B model at 4.5 bits is 7 × 4.5 ÷ 8 ≈ 3.9 GB of weights. Add the KV cache and about a gigabyte of overhead for the memory it actually occupies while running.
Why is Q4_K_M 4.5 bits and not 4?
K-quants mix precisions inside each block and keep some tensors at higher precision, so the effective rate lands above the nominal one. 4.5 bits per weight is the figure that matches real file sizes.
Is RAM the same as VRAM here?
The arithmetic is identical; the bandwidth is not. A model in system RAM runs at perhaps 50 GB/s against a card’s 500–1800 GB/s, so it fits but runs far slower.
Do I need extra space for the download?
Usually yes — some tools write a temporary file before moving it into place, so keep roughly double the final size free while pulling.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on memory
Will it fit
Context
Speed
Quantisation
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.