Kimi K3 Unveiled: 2.8 Trillion‑Parameter Open‑Source LLM Sets New Record

On Thursday, July 16, the Moon‑of‑Darkness project announced the launch of Kimi K3, a next‑generation open‑source foundation model with a parameter count of 2.8 trillion (2.8 兆). The model is immediately available through an API service and comprehensive developer documentation.

Kimi K3 is built around two novel components: Kimi Delta Attention (KDA), a mixed‑linear attention mechanism, and Attention Residuals (AttnRes). These structures improve long‑sequence information processing and deep‑network signal propagation, enabling native visual understanding and a context window of one million tokens.

The architecture expands the sparsity of Mixture‑of‑Experts (MoE) by integrating the Stable LatentMoE framework, allowing the model to activate only 16 out of 896 experts. Combined with optimized training recipes, this yields an overall scaling efficiency roughly 2.5 times higher than the previous K2 model, converting compute power into capability more effectively.

Several training innovations are introduced: Quantile Balancing allocates experts based on the quantiles of routing scores, eliminating heuristic updates and sensitive balance hyper‑parameters; Per‑Head Muon extends Muon optimization to each attention head independently; Sigmoid Tanh Unit (SiTU) and Gated MLA enhance activation control and attention selectivity. From the SFT stage onward, Kimi K3 employs quantization‑aware training with MXFP4 weights and MXFP8 activations, broadening hardware compatibility. Expert‑parallel training uses a fully balanced approach with static shapes and no host‑side synchronization, preventing throughput loss from uneven expert loads.

For inference, the team recommends deploying Kimi K3 on a supernode comprising 64 or more accelerators. The new attention design introduces challenges for traditional prefix‑caching; a compatible implementation has been contributed to the vLLM community, enabling competitive token‑price service even under long‑context conditions.

Independent evaluations reported by Xinhua and other media place Kimi K3’s overall intelligence near the frontier of closed‑source models, making it the largest open‑source model to date. While it still trails the strongest proprietary systems such as Claude Fable 5 and GPT‑5.6 Sol, it consistently outperforms all other open models in the same benchmark suite.

The model demonstrates strong long‑range coding abilities, handling extensive engineering tasks with minimal human supervision, understanding large codebases, and orchestrating terminal tools. It also excels at multimodal scenarios, leveraging screenshots and visual feedback to improve game development, front‑end work, and CAD pipelines, thereby advancing end‑to‑end knowledge‑work workflows.

Users can access Kimi K3 via kimi.com, the latest Kimi mobile app, the Kimi Work desktop client, Kimi Code, or the Kimi API. The default reasoning mode is set to “max” (extreme), with additional low and high modes planned for future updates. Full model weights will be released by July 27 2026, accompanied by a detailed technical report covering architecture, training, and evaluation.