The RTX 4090 pairs 24 GB of VRAM with 1,008 GB/s of memory bandwidth. Those two numbers, in that order, determine everything about what it can run and how quickly.
Of the 53 open-weight models we track, 53 fit entirely in this card's memory at some usable quantisation. The largest is Mixtral 8x22B Instruct at 39B parameters, which works because only 39B are active per token.
Highest-fidelity quantisation that fits in 24 GB, with an 8K context window:
| Model | Parameters | Best quant | VRAM used | Est. speed |
|---|---|---|---|---|
| Mixtral 8x22B Instruct | 39B / 39B active | Q3_K_M | 21.1 GB | ~47 tok/s |
| Qwen3.6 35B A3B | 35B | Q4_K_M | 23.6 GB | ~41 tok/s |
| Qwen3.5-35B-A3B | 35B | Q4_K_M | 23.6 GB | ~41 tok/s |
| Laguna XS 2.1 | 33B | Q4_K_M | 22.4 GB | ~43 tok/s |
| Qwen2.5 Coder 32B Instruct | 32B | Q4_K_M | 21.8 GB | ~45 tok/s |
| Qwen3 VL 32B Instruct | 32B | Q4_K_M | 21.8 GB | ~45 tok/s |
| Qwen3 32B | 32B | Q4_K_M | 21.8 GB | ~45 tok/s |
| Gemma 4 31B | 31B | Q4_K_M | 21.2 GB | ~46 tok/s |
| GLM 4.7 Flash | 30B | Q4_K_M | 20.6 GB | ~48 tok/s |
| Qwen3 30B A3B | 30B | Q4_K_M | 20.6 GB | ~48 tok/s |
| Muse Glimmer 30B | 30B | Q4_K_M | 20.6 GB | ~48 tok/s |
| Qwen3 30B A3B Instruct 2507 | 30B / 3B active | Q5_K_M | 23.3 GB | ~343 tok/s |
| Qwen3 Coder 30B A3B Instruct | 30B | Q4_K_M | 20.6 GB | ~48 tok/s |
| Qwen3 VL 30B A3B Instruct | 30B | Q4_K_M | 20.6 GB | ~48 tok/s |
| Nemotron 3.5 Lightning | 30B / 3B active | Q5_K_M | 23.2 GB | ~377 tok/s |
| Nemotron 3 Nano Omni | 30B | Q4_K_M | 20.6 GB | ~48 tok/s |
| Nemotron 3 Nano 30B A3B | 30B | Q4_K_M | 20.6 GB | ~48 tok/s |
| Cohere North Mini Code | 30B / 3B active | Q5_K_M | 23.2 GB | ~377 tok/s |
| Qwen3.8 27B | 27B | Q5_K_M | 22.8 GB | ~42 tok/s |
| Qwen3.6 27B | 27B | Q5_K_M | 22.8 GB | ~42 tok/s |
| Qwen3.5-27B | 27B | Q5_K_M | 22.8 GB | ~42 tok/s |
| Gemma 2 27B | 27B | Q5_K_M | 22.8 GB | ~42 tok/s |
| Gemma 4 26B A4B | 25B | Q5_K_M | 21.4 GB | ~45 tok/s |
| Mistral Small 3.1 24B | 24B | Q6_K | 23.2 GB | ~41 tok/s |
| Voxtral Small 24B 2507 | 24B | Q6_K | 23.2 GB | ~41 tok/s |
Generating a token requires reading the model's active weights out of memory once. That makes decoding a bandwidth problem, not a compute problem — which is why a card's headline TFLOPS number tells you almost nothing about how fast it will feel.
At 1,008 GB/s, the RTX 4090 can stream roughly 1,434 billion Q4 parameters per second. Divide that by a model's active parameter count and you have its ceiling in tokens per second. A 7B model lands near 205 tok/s; a 30B dense model near 48 tok/s. No amount of compute changes that ratio.
The buying rule this implies. Choose capacity for the model you want to run, then bandwidth for how it will feel. Two cards with identical VRAM can differ threefold in speed, and the spec sheet buries the number that decides it.
At 450W under sustained inference load, four hours a day works out to 1.8 kWh — about $9 a month at $0.17/kWh, or ₹432 at Indian residential rates. Size the PSU with headroom: 450W on the card means a 750W supply is the sensible floor once you add a CPU and transients.
| GPU | MSRP | VRAM | Bandwidth | Models it runs |
|---|---|---|---|---|
| RTX 4090 | $1,599 | 24 GB | 1,008 GB/s | 53 |
| RTX 3090 | $1,499 | 24 GB | 936 GB/s | 53 |
| RTX 5090 | $1,999 | 32 GB | 1,792 GB/s | 53 |
| RTX 5080 | $999 | 16 GB | 960 GB/s | 38 |
Read that table by column, not by row. The VRAM column tells you which models are even on the menu; the bandwidth column tells you how they will feel. Cards that look interchangeable on price routinely differ by a factor of two in one column and not the other, and which of those matters depends entirely on whether your target model fits.
At Q4_K_M, 24 GB holds roughly 40B parameters. Mixture-of-experts models change the arithmetic for speed but not for capacity — all experts still have to be resident.
For models up to about 40B parameters at Q4, yes. Its 1,008 GB/s of bandwidth is what sets the speed ceiling, and its 24 GB of capacity is what sets the size ceiling. Beyond that you are offloading to system RAM and losing most of the benefit.
Its rated board power is 450W, and sustained generation will sit near that. Inference load is steadier than gaming load, so plan for the full figure continuously rather than in bursts.
For capacity, largely yes — llama.cpp and vLLM both split layers across cards, so two 24 GB cards hold roughly 48 GB of model. For speed, no: layers run sequentially across the split, so you gain very little throughput on a single request.
They answer different questions. VRAM decides whether a model runs at all; bandwidth decides how fast it generates once it does. Buy enough capacity for your target model first, because no amount of bandwidth rescues a model that does not fit.