What chunk size and top-k should I use for RAG?
Top-k is not a number you choose freely. It is whatever is left of the context after the system prompt, the question and room to answer — divided by your chunk size.
The short answer
Computed from this tool’s default settings — context length and the rest as most people start. Change them below for your own case.
A 8,192 token window leaves 6,692 tokens for retrieval, so top-k 13 at 512-token chunks. Smaller chunks buy more distinct hits when the answer is spread across documents.
RAG Chunk Size Calculator
How many chunks fit in your context once the prompt takes its share.
A 8,192 token window leaves 6,692 tokens for retrieval, so top-k 13 at 512-token chunks. Smaller chunks buy more distinct hits when the answer is spread across documents.
| Chunk size | top-k | Tokens used | Recall |
|---|---|---|---|
| 128 | 52 | 6,656 | Good recall |
| 256 | 26 | 6,656 | Good recall |
| 512 | 13 | 6,656 | Good recall |
| 1,024 | 6 | 6,144 | Good recall |
| 2,048 | 3 | 6,144 | Tight |
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Context length | 8192 tokens | Anywhere from 1,024 to 131,072 tokens. |
| Chunk size | 512 tokens | Anywhere from 64 to 2,048 tokens. |
| Chunk overlap | 10 % | Anywhere from 0 to 50 %. |
| System prompt | 500 tokens | Your own figure in tokens, starting from 500. Change it to match what you actually run. |
| Room for the answer | 1000 tokens | Output shares the context window. Forgetting this is the usual cause of truncated answers. |
Real answers across context length
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Context length | Chunks you can retrieve | Context free for retrieval | Unique text retrieved | Reserved for output |
|---|---|---|---|---|
| 1,024 | top-k 0 | 0 tok | 0 tok | 1,000 tok |
| 33,536 | top-k 62 | 32,036 tok | 28,570 tok | 1,000 tok |
| 66,048 | top-k 126 | 64,548 tok | 58,061 tok | 1,000 tok |
| 98,560 | top-k 189 | 97,060 tok | 87,091 tok | 1,000 tok |
| 131,072 | top-k 253 | 129,572 tok | 116,582 tok | 1,000 tok |
After the system prompt and answer there is barely room for one chunk. Shrink the chunk size, trim the system prompt, or use a longer-context model.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
The context window has to hold the system prompt, the retrieved chunks, the question and the answer. Top-k is whatever is left after the other three, divided by the chunk size — not a number you pick freely.
Overlap buys recall at the cost of retrieving the same text twice. Ten to twenty percent is the usual range; past that you are spending context to read duplicates.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
Building RAG
Size chunks and top-k against a real window.
Truncated answers
Find the room you did not leave for output.
Tuning recall
Trade chunk size against number of hits.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set context length
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
4 further settings: chunk size, chunk overlap, system prompt, room for the answer. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
What chunk size and top-k should I use for RAG: common questions
The questions people ask about this, answered without hedging.
What chunk size is best for RAG?
For prose, 256–512 tokens retrieves precisely without losing the surrounding argument. Larger chunks bring more context per hit but fewer distinct hits, which hurts when the answer is spread across documents.
What is the best chunk size for RAG?
256–512 tokens for prose. Small enough to retrieve precisely, large enough to keep the surrounding argument intact. Code and tables often want larger chunks with structural boundaries rather than fixed sizes.
How much chunk overlap should I use?
Ten to twenty percent. It stops an answer being split across a boundary. Beyond that you are spending context to read the same sentences twice.
How many chunks should I retrieve?
As many as fit after reserving room for the output — usually five to ten. More retrieved context is not automatically better; irrelevant chunks actively distract.
Why are my answers getting cut off?
Because the retrieved context filled the window and left no room to generate. Input and output share the same budget, and reserving output space is the fix.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on context
Will it fit
Speed
Quantisation
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.