It Takes 59 Wafers to Hold a 3-Trillion-Parameter Model: Cerebras Shows Its Kimi K3 Math

Bigger models mean bigger hardware bills. In a technical post dated October 22, AI chipmaker Cerebras worked out what it would take to fit Moonshot AI's roughly 3-trillion-parameter Kimi K3 onto its wafer-scale chips and actually serve users with it: 59 wafers in total. That is not one machine's specification, but a whole hardware layout.

Where the number 59 comes from

Start with storing the model: Cerebras assumes each parameter stored in 4 bits, so 3 trillion parameters of weights alone come to about 1.5 terabytes; with 44 GB of on-chip memory per wafer, weights alone need a minimum of roughly 35 wafers before any room for user requests. Kimi K3 has 93 layers: 69 linear-attention layers and 24 dense-attention layers. Dense-attention layers store a KV cache that grows with request length, so Cerebras gives each one its own wafer; linear-attention layers go two per wafer across 35 wafers; the two add up to 59. The calculation also assumes an 8-bit KV cache and an average request of 256,000 tokens, and the company advertises output speeds of 1,000 to 2,000-plus tokens per second for multi-trillion-parameter models.

The important caveat: this is not a measured result yet

The line must be drawn clearly: Cerebras has not established whether the Kimi K3 layout was actually run, and has published no full benchmark conditions, measured power consumption, or clear count of simultaneous users the layout supports; the 1,000-to-2,000-plus tokens per second figure is the company's advertised speed for this class of model, not a measured result for this layout. The company itself concedes that batch size and concurrency alone do not establish inference efficiency, and that the real yardstick should be tokens per unit of power at the same output speed. Note also that Cerebras rounds Kimi K3 to about 3 trillion parameters, while Alibaba Cloud's model documentation lists 2.8 trillion with a one-million-token context window — a capability ceiling, not the same thing as the average request length assumed in the calculation.

The precedent for the method is Cerebras's earlier multi-system pipeline for GPT-5.6 Sol, advertised at up to 750 tokens per second; that is a precedent for the method, not a report card for Kimi K3. For the 59-wafer claim to stand, the next deliverable should be a measurement under the same assumptions: model and precision, request lengths and load, hardware count, and speed and power together.