Kimi K3 Unveiled: First Open‑Source 3‑Trillion‑Parameter Model with 1M Context

Kimi announced the release of Kimi K3, an open‑source large language model that reaches the 3‑trillion‑parameter scale, supports a 1 M token context window, and includes native visual‑understanding capabilities.

According to Arena.ai’s latest leaderboard, Kimi K3 ranks first in the front‑end code arena, far surpassing Claude Fable 5. In the Artificial Analysis Index, K3 obtains a score of 57, slightly below the closed‑source leader Claude Fable 5 (60) and GPT‑5.6 Sol (59), yet it remains well ahead of other open models.

The model’s architecture combines several novel techniques described in Kimi’s technical blog: Kimi Delta Attention (KDA) improves attention efficiency on long sequences; Attention Residuals (AttnRes) selectively retrieve representations from different depths; Stable LatentMoE expands total parameters to 2.8 T while activating only 16 of 896 experts per forward pass.

To maintain computational cost while scaling, Kimi K3 employs Quantile Balancing, which assigns experts based on the quantile of routing scores, reducing reliance on heuristic updates and sensitive hyper‑parameters. From the supervised fine‑tuning stage, the model uses quantization‑aware training with MXFP4 weights and MXFP8 activations to accommodate low‑precision hardware.

Additional innovations include the Per‑Head Muon optimizer, which extends the Muon optimizer to optimize each attention head independently, and activation‑control enhancements such as the Sigmoid Tanh Unit (SiTU) and Gated MLA, both boosting attention selection capabilities.

These architectural and engineering advances enable Kimi K3 to achieve roughly 2.5× the scaling efficiency of its predecessor K2, allowing stable training and deployment at the 2.8 T‑parameter scale.

Experimental results show that, while K3 still trails the strongest proprietary models, it consistently outperforms other evaluated models across benchmark suites and offers excellent cost‑performance. In internal evaluations, K3 (max) demonstrates continuous performance gains driven by real‑world knowledge‑work scenarios, highlighting its strength in end‑to‑end knowledge‑worker applications.

Kimi K3 is publicly available with a default “Max” inference mode; future updates will add “Low” and “High” modes for broader hardware compatibility.