Kimi K3 Is the Largest Open-Weight Model Ever — Can You Run It?

Moonshot AI just shipped Kimi K3, and at 2.8 trillion parameters it is the largest open-weight model ever released. It took the number-one spot on the Frontend Code Arena with 1,679 points — ahead of Claude Fable 5, GPT-5.6 Sol, and GLM-5.2 — and Moonshot has promised the full weights on Hugging Face by July 27. The obvious Runyard question: can you actually run a 2.8T model on your own hardware, and if not, what should you run instead?

Kimi K3 is a sparse mixture-of-experts (MoE) model with roughly 2.8 trillion total parameters, a 1-million-token context window, native vision, and always-on reasoning aimed at long-horizon coding and agent workloads. Moonshot launched the API and consumer app on July 16, 2026, and the model immediately made noise: it topped Arena.ai's Frontend Code Arena at 1,679 points and posted 88.3% on Terminal-Bench 2.1. Demand was heavy enough that Moonshot temporarily suspended new K3 subscriptions while it scaled serving capacity.

The architecture is where it gets interesting for local inference. K3 introduces Stable LatentMoE, a routing scheme with 896 experts that activates only 16 per token, plus two novelties — Kimi Delta Attention (KDA) for efficient long-sequence scaling and Attention Residuals (AttnRes) for selective depth-wise retrieval. Crucially, K3 was trained quantization-aware, with MXFP4 weights and MXFP8 activations, so the released weights are already low-precision rather than FP16 you have to squeeze down yourself.

In independent testing, Kimi K3 landed an Artificial Analysis Elo of 1547 on long-horizon knowledge work and used about 21% fewer output tokens than K2.6 for the same tasks — a real efficiency win, since output tokens are what you pay for in latency and, on an API, in dollars ($3 / $15 per million input/output tokens).

That is the headline: an open-weight model beating the best closed frontier models on a competitive coding benchmark. But a leaderboard score you can't run at home is just a spectator sport, so let's get to the hardware. It's worth pausing on why this matters beyond the numbers: an open-weight model at the top of a frontier leaderboard means the capability is no longer gated behind a single vendor's API and rate limits. Whether or not you personally can host it, the weights being public means researchers can inspect the routing, quantizers can compress it, and hosting competition can drive the per-token price down — the same dynamic that made earlier Kimi and DeepSeek models cheap to rent.

MoE is the reason a 2.8T model is even conceivable outside a hyperscaler. Only 16 of 896 experts fire for any given token, so the compute per token is closer to a mid-sized dense model than to a 2.8-trillion-parameter monster. That is great for throughput — but it does not save you memory. Every expert has to be resident somewhere the GPU can reach, because the router might pick any of them on the next token. This is the single most misunderstood fact about running MoE models locally, and it's worth internalizing before you plan a build. Our explainer on [how local LLM inference actually works](/blog/how-local-llm-inference-actually-works) walks through why loading, not compute, is usually the wall you hit first.

The MXFP4 quantization-aware training helps here. Because the weights were trained to tolerate 4-bit precision, the official release is already compact relative to an FP16 model of the same parameter count, and community GGUF ports should degrade gracefully. If you're fuzzy on what a 4-bit quant actually costs you in quality, our breakdown of [Q4_K_M vs Q5_K_M vs Q6_K vs Q8_0](/blog/q4-k-m-vs-q5-k-m-vs-q6-k-vs-q8-0-quantization-quality-loss) shows how small the real-world hit is on general tasks — and how quantization tends to magnify a model's existing weaknesses rather than invent new ones.

For nearly everyone reading this, the honest answer today is no — and it will still be no on July 28. Community estimates put the loaded footprint at roughly 594 GB in its native precision, which is why you need on the order of eight H100s (8 x 80 GB = 640 GB) just to hold the weights. Aggressive community GGUF quants will help: a Q4-class build is estimated around 300-400 GB, and even a punishing 2-bit-class quant lands near 200 GB. All of those numbers are still far beyond any single desktop GPU.

A 24 GB RTX 4090 cannot load K3 in any quantization. A 512 GB CPU-RAM workstation with expert offloading via llama.cpp could in principle stream a heavy quant, but you'd be measuring speed in seconds per token, not tokens per second. If your goal is a genuinely usable local coding model in the Kimi family, the right move is Kimi K2.7 Code, which runs on a single 24 GB GPU with quantization. For a wider view of what fits your box, see our guide to the [best local LLMs for coding in 2026](/blog/best-local-llms-for-coding-2026) and [how much VRAM you actually need](/blog/how-much-vram-to-run-local-llms).

So treat K3 the way you'd treat a new supercar: worth understanding, occasionally worth renting on a cloud GPU, but not something that lives in your garage. The practical play once weights drop is to run K2.7 Code locally for day-to-day work and reserve K3 for the hard problems you'll route to a rented 8x H100 node or a hosted open-model provider.

There is a nuance MoE gives you that dense giants don't: because only 16 of 896 experts fire per token, the bottleneck is memory bandwidth to wherever the experts live, not raw FLOPs. That is exactly why llama.cpp's expert-offloading trick — keeping the sparse feed-forward experts in cheap system RAM while the dense attention layers sit on the GPU — makes a 2.8T model technically loadable on a high-RAM workstation, even if the tokens-per-second are painful. If you have 512 GB of DDR5 and patience, a 2-bit community quant plus the '-ot' flag in the code block below is the closest thing to a home-lab K3. For anything interactive, though, rent the hardware or run a smaller model — the economics are covered in our comparison of [local LLM vs OpenAI API cost](/blog/local-llm-vs-openai-api-cost).

# TODAY: run the local-friendly sibling, Kimi K2.7 Code, on a single 24GB GPU
# (Unsloth ships dynamic GGUF quants that fit consumer hardware)
ollama pull hf.co/unsloth/Kimi-K2.7-Code-GGUF:Q4_K_M
ollama run hf.co/unsloth/Kimi-K2.7-Code-GGUF:Q4_K_M

# PREP FOR KIMI K3 (weights land ~July 27). Grab the community GGUF once it's up:
pip install -U huggingface_hub
huggingface-cli download unsloth/Kimi-K3-GGUF \
  --include "*Q2_K_XL*" --local-dir ./kimi-k3

# Run a giant MoE in llama.cpp by keeping experts on CPU RAM and only
# offloading attention/dense layers to the GPU (-ot = override tensor placement):
llama-cli -m ./kimi-k3/Kimi-K3-Q2_K_XL-00001-of-000NN.gguf \
  -ngl 99 -c 8192 \
  -ot "\.ffn_.*_exps\.=CPU" \
  -p "Write a Python function to parse a GGUF header."

Kimi K3 wasn't the only thing moving. A few other releases worth your attention if you run models locally:

• ▸Thinking Machines Lab — Mira Murati's startup — shipped its first model, Inkling, on July 15 as an open-weight MoE with 975B total parameters and about 41B active per token, a rare open release from a US frontier lab. (Source: TechCrunch, techcrunch.com/2026/07/15)
• ▸DeepSeek is lining up a stable DeepSeek V4 release on July 24, and DeepSeek-V4-Flash is already runnable via Unsloth (unsloth>=2026.7.2) with Thinking toggles and an improved chat template. (Source: Unsloth docs, unsloth.ai/docs/new/changelog)
• ▸Unsloth's Qwen3.6-27B dynamic GGUFs now run and fine-tune in about 23 GB of RAM, making it one of the strongest mid-sized models you can actually keep on a desktop. (Source: huggingface.co/unsloth/Qwen3.6-27B-GGUF)
• ▸Unsloth Studio added export to NVFP4, FP8, and imatrix GGUFs after training, a built-in llama-swap API mode, and 3-5x faster MoE training — useful plumbing for anyone quantizing their own checkpoints. (Source: Unsloth changelog, unsloth.ai/docs/new/changelog)

Before you download a 300 GB quant or spin up a cloud node, find out what your machine can honestly handle. Drop your GPU into the [Runyard VRAM Calculator](/tools/vram-calculator) to see which quant of K2.7 Code — or any model — fits your VRAM, context length, and target speed. If you're weighing K3 against smaller open-weight coders, [Runyard Compare](/compare) puts them head to head, and [Model Radar](/) surfaces the best model your hardware can run today. The largest open-weight model ever is a milestone worth celebrating; running the right model for your box is what actually ships code.