What llama.cpp flags should I use?
llama.cpp has dozens of flags and three that decide whether the thing runs well. This produces a command sized to your card instead of the usual step-down-and-retry loop.
The short answer
Computed from this tool’s default settings — your hardware and the rest as most people start. Change them below for your own case.
Everything fits on RTX 4090 — offload all layers and keep the full 8,192 token context.
llama.cpp Flags Generator
A complete, copy-pasteable command sized to your card and context.
Everything fits on RTX 4090 — offload all layers and keep the full 8,192 token context.
llama-server \ -m model-Q4_K_M.gguf \ --n-gpu-layers 999 \ --ctx-size 8192 \ --flash-attn \ --port 8080
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Your hardware | RTX 4090 · 24 GB | The machine the model runs on. Usable memory decides what fits and memory bandwidth decides how fast it runs, so this moves every figure below. |
| Model size | 8B | 9 options, from 1B to 405B. |
| Quantisation | Q4_K_M — 4.5 bits/weight | 5 options, from Q8_0 — 8.5 bits/weight to Q3_K_M — 3.5 bits/weight. |
| Context length | 8192 tokens | Anywhere from 1,024 to 131,072 tokens. |
Real answers across model size
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Model size | GPU layers | Total needed | Your memory | Estimated layers |
|---|---|---|---|---|
| 1B | 999 (all) | 2.05 GB | 22.1 GB | 17 |
| 3B | 999 (all) | 3.54 GB | 22.1 GB | 24 |
| 7B | 999 (all) | 6.24 GB | 22.1 GB | 32 |
| 8B | 999 (all) | 6.89 GB | 22.1 GB | 33 |
| 14B | 999 (all) | 10.7 GB | 22.1 GB | 40 |
Everything fits on RTX 4090 — offload all layers and keep the full 8,192 token context.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
The three flags that matter are --n-gpu-layers, --ctx-size and --flash-attn. Layers decide how much of the model sits on the GPU; context decides the KV cache; flash attention reduces peak cache memory on Ampere and newer cards at no quality cost.
Layer counts are estimated from parameter count, since models do not publish them uniformly. If the command fails to load, drop --n-gpu-layers by two or three and try again — that is the standard tuning loop.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
First run
Start from a command that works rather than defaults.
After an OOM
Get a configuration that fits.
Serving
Set context and layers deliberately.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set your hardware
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
3 further settings: model size, quantisation, context length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
What llama.cpp flags should I use: common questions
The questions people ask about this, answered without hedging.
What llama.cpp flags matter most?
--n-gpu-layers, --ctx-size and --flash-attn. Layers decide how much sits on the GPU, context decides the KV cache, and flash attention cuts peak memory at no quality cost.
What does --flash-attn do?
Computes attention in tiles rather than materialising the full matrix, which reduces peak memory and helps more as context grows. There is no quality trade-off, so leave it on where it is supported.
Should I set --n-gpu-layers 999?
Yes when everything fits — it is the shorthand for offloading every layer. Only use a specific number when the model is too large and you are deliberately splitting.
How do I serve an OpenAI-compatible API?
llama-server exposes one on the port you pass with --port, so most OpenAI client libraries work by changing the base URL.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on runtime & errors
Will it fit
Context
Speed
Quantisation
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.