770B Parameters Under Apache 2.0. What Does It Take to Run?

Hy4 is a 770-billion-parameter mixture-of-experts model with 49B active per token, released under Apache 2.0. Open weights at frontier scale, which raises an obvious question: can anyone actually run it?

Barely, and only at the edges.

Capacity follows total parameters, not active ones. Every expert has to be resident because the router can select any of them on any token. So all 770B must be in memory:

• ▸Q8_0 — about 818 GB of weights.
• ▸Q6_K — about 635 GB.
• ▸Q4_K_M — about 433 GB.
• ▸Q3_K_M — about 337 GB.
• ▸2-bit — about 192 GB, at real quality cost.

At Q4_K_M, nothing available to a consumer. A 512 GB Mac Studio M5 Ultra gives about 384 GB usable, which is short of 433. Eight RTX 5090s give roughly 230 GB between them, and cost more than the Mac.

Drop to Q3_K_M and 337 GB fits inside that 384 GB — so a maxed Mac Studio is, as far as we can tell, the only single machine a person can buy that runs this model at a quantisation worth using.

Open weights at this scale change what is possible for anyone with server hardware, and they set a ceiling that hosted providers have to price against. You do not need to run it for it to affect what you pay for inference.

It also sharpens the MoE lesson that keeps catching people out. This model will be described as “49B active” in a lot of coverage. That figure describes its speed. Its memory requirement is 770B, and no amount of sparsity changes that.

Most people who want to try a model this size will rent rather than buy, and the arithmetic there is much friendlier. A machine with enough memory to hold 337 GB of weights runs on community clouds at a few dollars an hour.

At, say, $3 an hour, an afternoon of experimentation costs less than a takeaway and you do not own a depreciating asset. Compare that to a maxed Mac Studio, where you would need to run the model for thousands of hours before ownership made financial sense — and by then there will be a better open model.

Worth asking before planning a purchase around it. More parameters generally means better reasoning, broader knowledge and more reliable instruction following. It does not mean better at everything, and the gap is not proportional to the parameter count.

A well-trained 27B released this year will beat a mediocre 200B from last year on most tasks. And for a specific narrow job — summarising, classification, structured extraction, code completion — a small model that fits your card comfortably usually wins on practicality, because you can run it constantly without thinking about it.

The honest case for a model this size is work where breadth genuinely matters: open-ended research, long multi-step reasoning, or anything where you cannot predict what knowledge will be needed.

To fit 770B into any consumer machine you have to quantise hard, and hard quantisation is where quality actually degrades:

• ▸Q4_K_M at 433 GB — the level we would normally recommend, and out of reach.
• ▸Q3_K_M at 337 GB — fits a 512 GB machine, with the documented quality cost of dropping below 4 bits.
• ▸2-bit at 192 GB — fits far more hardware, and is where models often start producing noticeably worse reasoning.

So the realistic choice is a 770B model at a quantisation nobody recommends, against a 70B or 110B model at a quantisation that is fine. That second option is very often the better model in practice, which is the uncomfortable conclusion for anyone excited by the parameter count.

Why does a MoE model need all its parameters in memory?

Because the router picks experts per token and can select any of them. There is no way to predict which experts a conversation will need, so all of them have to be resident. Sparsity buys speed, never capacity.

Can I run part of it on GPU and part on CPU?

Technically yes, and it will be extremely slow. Every token passes through the full stack, so the offloaded portion dominates. At this size the offload would be most of the model.

What about a server with lots of ordinary RAM?

A 12-channel server platform gives roughly 576 GB/s of bandwidth and can be configured with far more than 512 GB. It will hold the model at Q4 and run it at maybe half the speed of unified memory. If you are buying hardware for models this size rather than for a laptop, that is the more sensible shape.