Why does a mixture-of-experts model need so much memory?
A mixture-of-experts model is quoted by its active parameters and sized by its total ones. Confusing the two is the single most common mistake in local AI sizing.
The short answer
Computed from this tool’s default settings — total parameters and the rest as most people start. Change them below for your own case.
It needs 135.5 GB but RTX 4090 has 22.1 GB. The 22B active figure is why it would be fast, not a reason it will fit — all 235B must be resident.
MoE vs Dense Memory Calculator
Capacity follows total parameters; speed follows the active ones.
It needs 135.5 GB but RTX 4090 has 22.1 GB. The 22B active figure is why it would be fast, not a reason it will fit — all 235B must be resident.
| Quantity | Value | Driven by |
|---|---|---|
| Memory needed | 135.5 GB | Follows total (235B) |
| If you assumed active | 15.7 GB | Wrong by 119.8 GB |
| Speed | 65 tok/s | Follows active (22B) |
| Dense model of same size | 6.1 tok/s | 10.7× slower |
What each setting changes
Every input moves the result for a reason. This is what each one does and where to find the value for your own machine.
| Setting | Default | What it changes |
|---|---|---|
| Total parameters | 235 B | Your own figure in B, starting from 235. Change it to match what you actually run. |
| Active parameters per token | 22 B | The figure marketing quotes. It sets speed, not memory. |
| Quantisation | Q4_K_M — 4.5 bits/weight | 5 options, from Q8_0 — 8.5 bits/weight to Q3_K_M — 3.5 bits/weight. |
| Your hardware | RTX 4090 · 24 GB | The machine the model runs on. Usable memory decides what fits and memory bandwidth decides how fast it runs, so this moves every figure below. |
| Context length | 8192 tokens | Anywhere from 1,024 to 131,072 tokens. |
Real answers across quantisation
The same calculation run at a range of settings, with everything else left at its default. These are computed by the tool itself, not written by hand.
| Quantisation | Does not fit | Your memory | Memory (total params) | Speed (active params) |
|---|---|---|---|---|
| Q8_0 — 8.5 bits/weight | 253.0 GB | 22.1 GB | 253.0 GB | 34 tok/s |
| Q6_K — 6.6 bits/weight | 197.2 GB | 22.1 GB | 197.2 GB | 44 tok/s |
| Q5_K_M — 5.7 bits/weight | 170.7 GB | 22.1 GB | 170.7 GB | 51 tok/s |
| Q4_K_M — 4.5 bits/weight | 135.5 GB | 22.1 GB | 135.5 GB | 65 tok/s |
| Q3_K_M — 3.5 bits/weight | 106.1 GB | 22.1 GB | 106.1 GB | 84 tok/s |
It needs 253.0 GB but RTX 4090 has 22.1 GB. The 22B active figure is why it would be fast, not a reason it will fit — all 235B must be resident.
How this is calculated
No lookup tables and no invented constants. Here is the arithmetic, so you can check it against your own numbers.
A mixture-of-experts model routes each token through a small subset of its experts, but the router can pick any of them, so every expert must be resident. Memory therefore follows total parameters while throughput follows active parameters — the two numbers answer different questions.
This is the single most common mistake in local-AI sizing: a 235B model with 22B active is quoted as "22B" and then fails to load on a card that would run a dense 22B comfortably.
These are well-founded engineering estimates, not benchmark results. Your quantisation, runtime and context length all move the real number, and usable memory is an assumption rather than a specification. See the full methodology for every assumption behind these figures.
Who this is for
The three situations that bring people to this calculation.
MoE will not load
See why the active-parameter figure misled you.
Comparing architectures
Weigh MoE speed against its memory cost.
Planning
Size hardware for a MoE model correctly.
How to use this calculator
Four steps, no account, nothing leaves your browser.
Set total parameters
Start at the top of the panel. Every figure recalculates as you change it — there is no submit button, because watching the number move is the point.
Adjust the rest to match your setup
4 further settings: active parameters per token, quantisation, your hardware, context length. Defaults are the common case, so change only what differs for you.
Read the headline, then the table
The large figure answers the question. The table underneath shows how the answer changes across nearby settings, which is usually where the decision actually gets made.
Check it against the method
The arithmetic is written out above. If a number looks wrong for your hardware, the assumptions are the first place to look — usable memory and quantisation are the two that vary most.
Why does a mixture-of-experts model need so much memory: common questions
The questions people ask about this, answered without hedging.
Can I run a 235B MoE model on a 24GB card?
No. Even at 4-bit, 235B of weights is roughly 132 GB, and all of it has to be resident. The 22B active figure tells you it will be fast once loaded — not that it will fit.
Why does a 235B MoE with 22B active need so much memory?
Because the router can select any expert on any token, so all 235B of weights must be resident. The 22B figure describes how much is read per token — it sets speed, not memory.
Can I run a MoE model on a small card?
Only if the total parameters fit. There is no way to load a subset of experts, since the router may need any of them at any step.
Are MoE models worth it locally?
When you have capacity but not bandwidth, yes — they deliver large-model quality at small-model speed. On a memory-limited card, a dense model of the same footprint is usually stronger.
How do I read a MoE model name?
Two numbers, usually written total/active — 235B-A22B means 235B total with 22B active. Size your hardware on the first and expect speed from the second.
Related local AI tools
All 50 run on the same arithmetic, so answers across them agree.
More on memory
Will it fit
Context
Speed
Quantisation
Runtime & errors
Cost
Fine-tuning
Searches this page answers
Different ways of asking the same question, all resolved above.
Now find the models that fit
Sizing is only half the problem. Model Radar takes your hardware and shows which models actually run on it, ranked by what they are good at — the same arithmetic as this page, applied to every model worth running.