Which GGUF file should I download?
A single model on Hugging Face ships as a dozen GGUF files with cryptic suffixes. Only one of them is right for your card, and the difference between them is size, not features.
The short answer
Computed from this tool’s default settings — your hardware and the rest as most people start. Change them below for your own case.
Take the Q8_0 build — about 8.50 GB on disk. Use --include so you pull one file rather than the whole repository.
GGUF Variant Chooser
File sizes for every variant, and which one your hardware holds.
Take the Q8_0 build — about 8.50 GB on disk. Use --include so you pull one file rather than the whole repository.
| File | Size on disk | Memory in use | Verdict |
|---|---|---|---|
| *Q8_0*.gguf | 8.50 GB | 10.9 GB | Download this |
| *Q6_K*.gguf | 6.60 GB | 8.99 GB | Download this |
| *Q5_K_M*.gguf | 5.70 GB | 8.09 GB | Download this |
| *Q4_K_M*.gguf | 4.50 GB | 6.89 GB | Download this |
| *Q3_K_M*.gguf | 3.50 GB | 5.89 GB | Download this |
huggingface-cli download <repo> \ --include "*Q8_0*.gguf" --local-dir ./models
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Your hardware | RTX 4090 · 24 GB | The machine the model runs on. Usable memory decides what fits and memory bandwidth decides how fast it runs, so this moves every figure below. |
| Model size | 8B — Llama 3.1 8B class | Capacity follows total parameters, even for mixture-of-experts models. |
| Context length | 8192 tokens | The KV cache grows linearly with this. It is the biggest lever you have. |
Real answers across model size
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Model size | Download | File size | Memory in use |
|---|---|---|---|
| 1B — Llama 3.2 1B class | Q8_0 | 1.06 GB | 2.55 GB |
| 7B — Mistral / Qwen 7B class | Q8_0 | 7.44 GB | 9.74 GB |
| 14B — Qwen 14B class | Q8_0 | 14.9 GB | 17.7 GB |
| 32B — Qwen 32B class | Q4_K_M | 18.0 GB | 21.8 GB |
| 120B — gpt-oss 120B class | None fit |
Take the Q8_0 build — about 1.06 GB on disk. Use --include so you pull one file rather than the whole repository.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
A GGUF repository usually holds every quantisation of a model, and the files are large. Downloading the wrong one wastes both bandwidth and disk, and cloning the whole repository can pull tens of gigabytes you will never use.
The file on disk is the weights only. Running it also needs the KV cache for your context plus about a gigabyte of runtime overhead, which is why the "total" column below is larger than the download.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
At the download page
Pick the one file worth pulling.
Low on memory
Find the largest variant that still fits.
Comparing repos
Understand what the suffixes actually mean.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set your hardware
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
2 further settings: model size, context length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
Which GGUF file should I download: common questions
The questions people ask about this, answered without hedging.
What do the GGUF suffixes mean?
Q is quantised, the number is the nominal bits per weight, K marks the K-quant scheme, and S/M/L is the size variant within that level. Q4_K_M is a 4-bit K-quant, medium.
Which GGUF file should I download?
The largest one that fits your memory with room for the KV cache, which is usually Q4_K_M or Q5_K_M. Downloading a bigger file you cannot load wastes an hour.
What is an imatrix quant?
A build that uses an importance matrix from calibration data to decide which weights to keep precise. It generally beats a plain quant of the same size, especially at the low end, at no cost to you.
Are split GGUF files a problem?
No. Large models are published in parts; modern llama.cpp and Ollama load them by pointing at the first part, and there is no need to join them by hand.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on quantisation
Will it fit
Context
Speed
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.