Editorial Team
3 min read
Moonshot AI opens Kimi K3 weights, but deployment remains a supernode job
Moonshot AI has released the weights and technical report for Kimi K3, its 2.8T-parameter mixture-of-experts model. The release gives researchers and operators a deployable checkpoint and more of its training infrastructure, but Moonshot still recommends large supernodes and its performance claims are not a uniform, like-for-like comparison.
- Moonshot AI has released Kimi K3's full weights and technical report under the Kimi K3 License.
- The model has 2.8T total parameters but activates 104B per token; Moonshot recommends large supernodes for deployment.
- The release includes newly open-sourced training-infrastructure components, while its efficiency and benchmark claims remain company claims with non-uniform comparison conditions.
Moonshot AI, the developer of the Kimi model family, has completed the staged Kimi K3 rollout by publishing the model’s weights and technical report. The 2.8T-parameter mixture-of-experts model is now available for research and deployment under the Kimi K3 License, rather than only through Moonshot’s own service.
That is a meaningful expansion of access to Moonshot’s flagship Kimi model. It is not evidence that the model is easy to run outside a large operator: a report on the release says Moonshot recommends large supernode configurations because of its size and infrastructure requirements.
Moonshot AI reports 896 experts in Kimi K3 and 16 selected per token (company-reported). Source: GitHub.
A sparse model with a large deployment footprint
Kimi K3 is an MoE, or mixture-of-experts, model: it contains 896 experts but selects 16 for a token, with 104B activated parameters from 2.8T total parameters. In other words, about 1.8% of its named experts are selected for each token. That sparsity describes the model’s active computation; it does not change Moonshot’s stated recommendation of large supernodes.
The model release lists 93 layers, a 1,048,576-token context length, Kimi Delta Attention and Attention Residuals, and a 401M-parameter MoonViT-V2 vision encoder. Its stated input modalities are text and images. Moonshot also lists MXFP4 weights and MXFP8 activations, saying it applied quantization-aware training from the supervised-fine-tuning stage onward.
K3 follows the Kimi K2 generation. Moonshot says its Stable LatentMoE framework, Kimi Delta Attention and Attention Residuals deliver roughly 2.5 times K2’s overall scaling efficiency. That is a company characterization of how efficiently compute becomes capability, not a disclosed price, memory requirement or operating-cost comparison.
