Qwen3 30B A3B Instruct 2507 is a 30B-parameter mixture-of-experts model that activates only 3B parameters per token. That 9× gap is the single most misunderstood number in local AI: it decides your speed, but it does not shrink the download.
The short answer: at Q4_K_M, the quantisation most people actually run, Qwen3 30B A3B Instruct 2507 occupies roughly 16.9 GB of weights and needs about 18.8 GB of total VRAM once you add an 8K context window and runtime overhead. That puts it within reach of a RX 7900 XTX (24 GB) and anything larger.
Almost every listing will describe this as a 30B model, and almost every reader will conclude they need a datacentre. Both halves of that are half-true, and getting the distinction right is the difference between buying the correct GPU and buying twice as much as you need.
A mixture-of-experts model splits its feed-forward layers into many specialised experts and routes each token to a small subset. For Qwen3 30B A3B Instruct 2507, that means:
The practical consequence. Qwen3 30B A3B Instruct 2507 gives you the knowledge of a 30B model at the speed of a 3B one. What it will not do is fit in less memory. If you are choosing hardware, size the card against the 30B figure and set your speed expectations from the 3B one — reversing those two is the most common and most expensive mistake in local inference.
Quantisation trades precision for memory. Each row is Qwen3 30B A3B Instruct 2507 at 8K context, including the KV cache and ~1 GB of runtime overhead:
| Quant | Weights | Total VRAM | Quality |
|---|---|---|---|
Q8_0 | 31.9 GB | 33.8 GB | Near-lossless. Use when VRAM is free. |
Q6_K | 24.8 GB | 26.6 GB | Very close to FP16; the safe default if it fits. |
Q5_K_M | 21.4 GB | 23.3 GB | Mild loss, noticeably smaller. |
Q4_K_M | 16.9 GB | 18.8 GB | The sweet spot most people run. |
Q3_K_M | 13.1 GB | 15.0 GB | Visible degradation. Fallback only. |
Below Q4 the returns turn sharply negative. Q3_K_M saves 3.8 GB over Q4_K_M but introduces errors you will notice in structured output and long-form reasoning. If Q4 does not fit, a smaller model at Q5 or Q6 almost always beats this one at Q3.
Every current consumer card, against Qwen3 30B A3B Instruct 2507 at Q4_K_M. Throughput is estimated from memory bandwidth against the 3B active slice:
| GPU | VRAM | Bandwidth | Runs it? | Est. speed |
|---|---|---|---|---|
| RTX 5090 | 32 GB | 1,792 GB/s | Yes — Q6_K | ~527 tok/s |
| RTX 4090 | 24 GB | 1,008 GB/s | Yes — Q5_K_M | ~343 tok/s |
| RTX 3090 | 24 GB | 936 GB/s | Yes — Q5_K_M | ~318 tok/s |
| RX 7900 XTX | 24 GB | 960 GB/s | Yes — Q5_K_M | ~327 tok/s |
| RTX 5080 | 16 GB | 960 GB/s | Yes — Q3_K_M | ~532 tok/s |
| RTX 5070 Ti | 16 GB | 896 GB/s | Yes — Q3_K_M | ~496 tok/s |
| RTX 5060 Ti 16GB | 16 GB | 448 GB/s | Yes — Q3_K_M | ~248 tok/s |
| RTX 4080 SUPER | 16 GB | 736 GB/s | Yes — Q3_K_M | ~408 tok/s |
| RTX 4070 Ti SUPER | 16 GB | 672 GB/s | Yes — Q3_K_M | ~372 tok/s |
| RTX 4060 Ti 16GB | 16 GB | 288 GB/s | Yes — Q3_K_M | ~160 tok/s |
| RX 9070 XT | 16 GB | 645 GB/s | Yes — Q3_K_M | ~357 tok/s |
| RTX 5070 | 12 GB | 672 GB/s | Needs offload | — |
| Arc B580 | 12 GB | 456 GB/s | Needs offload | — |
Notice that VRAM and bandwidth do not move together. The RTX 4060 Ti 16GB holds as much as an RTX 4080 SUPER but reads it at 288 GB/s against 736 GB/s, so it will load this model and then generate at roughly a third the speed. Capacity decides whether it runs; bandwidth decides whether you enjoy using it.
Fastest path, using Ollama:
ollama run qwen3-30b-a3b-instruct-2507For control over quantisation and context, pull the GGUF directly and serve it with llama.cpp:
huggingface-cli download Qwen/Qwen3-30B-A3B-Instruct-2507 \
--include "*Q4_K_M*.gguf" --local-dir ./models
llama-server -m ./models/*Q4_K_M*.gguf \
--n-gpu-layers 999 \
--ctx-size 8192--n-gpu-layers 999 pushes everything onto the GPU; lower it until the model loads if you are short on memory. --ctx-size is worth tuning deliberately — this model supports up to 262K tokens, but the KV cache grows with it, and asking for the full window when you only need 8K can cost you several gigabytes for nothing.
About 18.8 GB at Q4_K_M with an 8K context window: 16.9 GB of weights plus KV cache and roughly 1 GB of runtime overhead. At Q8_0 it needs 33.8 GB, and at Q3_K_M it comes down to 15.0 GB.
The RX 7900 XTX at 24 GB is the least expensive card that holds it at Q4_K_M, at around $999.
Not entirely. It needs 18.8 GB at Q4_K_M, so an 8 GB card has to offload layers to system RAM, which cuts speed substantially. A smaller model at a higher quantisation will give better results on that hardware.
The weights are published openly on Hugging Face as Qwen/Qwen3-30B-A3B-Instruct-2507, but the licence is set by the model's publisher and varies — some are Apache 2.0 or MIT, others carry usage restrictions or revenue thresholds. Check the licence file on the model card before deploying it commercially.
Measurably, but far less than people expect down to Q4. Q6_K is close enough to FP16 that differences are hard to detect; Q4_K_M costs a small amount of accuracy on reasoning and structured output; below Q4 the degradation becomes obvious. Running a larger model at Q4 generally beats a smaller one at Q8.
These figures assume the whole model sits in VRAM and the GPU sustains its rated bandwidth. Real throughput drops if any layers are offloaded to system RAM, if your context is long enough that attention starts to dominate, or if the card is thermally throttling. Prompt processing is also compute-bound rather than bandwidth-bound and follows different limits.
Budget 19 GB for the Q4_K_M GGUF, or 35 GB for Q8_0. Download the specific quantisation you want rather than the whole repository — most GGUF repos hold every variant, and cloning all of them wastes a great deal of space.