What batch size can I fine-tune with?
Once the model is loaded, whatever memory is left decides your batch size — and batch size is what determines whether a training run takes hours or days.
The short answer
Computed from this tool’s default settings — model size and the rest as most people start. Change them below for your own case.
About 146 samples per step at 512 tokens. Use gradient accumulation to reach a larger effective batch without more memory — 32 to 64 effective is a common target.
Fine-Tune Batch Size Calculator
How much memory is left for activations after the model, and what that buys.
About 146 samples per step at 512 tokens. Use gradient accumulation to reach a larger effective batch without more memory — 32 to 64 effective is a common target.
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Model size | 8B | 9 options, from 1B to 405B. |
| Your hardware | RTX 4090 · 24 GB | The machine the model runs on. Usable memory decides what fits and memory bandwidth decides how fast it runs, so this moves every figure below. |
| Sequence length | 512 tokens | Activation memory scales with this as well as with batch size. |
Real answers across model size
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Model size | Batch size that fits | QLoRA base | Left for activations | Per sample |
|---|---|---|---|---|
| 1B | 571 | 1.20 GB | 17.6 GB | 0.03 GB |
| 3B | 286 | 3.50 GB | 15.3 GB | 0.05 GB |
| 7B | 169 | 5.00 GB | 13.8 GB | 0.08 GB |
| 8B | 146 | 6.00 GB | 12.8 GB | 0.09 GB |
| 14B | 89 | 8.50 GB | 10.3 GB | 0.11 GB |
About 571 samples per step at 512 tokens. Use gradient accumulation to reach a larger effective batch without more memory — 32 to 64 effective is a common target.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
QLoRA quantises the base model to 4-bit and trains small adapters on top, so the card that runs a model can usually train it too. LoRA keeps the base at 16-bit and trains the same adapters, costing roughly three to four times more memory. A full fine-tune updates every weight and carries gradients plus optimiser state — about 16 bytes per parameter.
The QLoRA and LoRA figures are Unsloth’s published requirements, interpolated between the sizes they list rather than extrapolated from a formula. They assume a short sequence length and a small batch; longer sequences and larger batches push the real number up.
Activation memory grows with batch size times sequence length. The estimate below assumes gradient checkpointing, which trades recomputation for memory and is what makes larger batches possible at all. Leave 15–20% headroom: at the edge, a single long example will push a run over.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
Setting up a run
Pick a batch size that will not crash at step 400.
Long sequences
See what a longer sequence length costs.
Tuning throughput
Balance real batch against accumulation steps.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set model size
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
2 further settings: your hardware, sequence length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
What batch size can I fine-tune with: common questions
The questions people ask about this, answered without hedging.
What batch size should I use for fine-tuning?
The largest that fits with headroom, then use gradient accumulation to reach an effective batch of 32 to 64. Effective batch is what affects training quality; the per-step batch only affects speed.
What is gradient accumulation?
Running several small batches and summing their gradients before stepping the optimiser. It gives the statistical behaviour of a large batch at the memory cost of a small one, in exchange for time.
Why does sequence length matter so much?
Activation memory scales with batch size times sequence length. Doubling the sequence halves the batch you can fit, which is why long-context fine-tuning is expensive.
Should I use gradient checkpointing?
Almost always. It recomputes activations during the backward pass instead of storing them, trading roughly 30% more time for a large memory saving — which usually buys back more than 30% through a bigger batch.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on fine-tuning
Will it fit
Context
Speed
Quantisation
Runtime & errors
Cost
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.