Kimi K3: The World’s First 3‑Trillion‑Parameter Open‑Source Model

Kimi K3, released by Moonshadow, features 2.8 trillion parameters, making it the first open‑source model of this scale. Its performance on long‑context programming, knowledge work, and reasoning approaches that of leading closed‑source models Claude Fable 5 and GPT‑5.6 Sol.

In the Frontend Code Arena, Kimi K3 scores 1679, surpassing Claude Fable 5 and leading other benchmarks, while consuming fewer tokens than its competitors, especially on the BrowseComp task.

The model demonstrates strong long‑range coding abilities: with minimal supervision it can sustain extensive engineering projects, understand large codebases, operate terminal tools, and combine software engineering with visual reasoning for game development, frontend work, and CAD.

To evaluate kernel‑optimization skills, the team built a GPU‑kernel arena where each model runs in an isolated sandbox for up to 24 hours, analyzing existing implementations, rewriting kernels, and repeatedly validating performance.

Kimi K3 also created MiniTriton, a compact Triton‑style compiler built on MLIR with a tile‑level intermediate representation and a full pass pipeline to PTX. In Roofline benchmarks MiniTriton matches or exceeds Triton and torch.compile, sometimes outperforming Triton.

The team fabricated a 4 mm² ASIC prototype using open‑source EDA tools and the Nangate 45 nm library. The chip integrates 1.46 M standard cells, 0.277 MB SRAM, and an INT4 MAC array, achieving 100 MHz timing and sustaining over 8 700 tokens / s during a 48‑hour autonomous agent run.

Four multimodal demos showcase Kimi K3’s capabilities: a 3D simulation of a Long March‑10 rocket launch and recovery, an open‑world 3D game, a 3D GBA emulator, and a reconstruction of the Kerr‑Newman black‑hole geometry.

In knowledge‑work scenarios, K3 generated a 42‑year ASIC industry report after more than 1 200 recursive self‑improvements, performing 2 800 web searches, processing 11 000 pages of data, and producing custom charts, animations, and interactive narratives. It also analyzed 391 gravitational‑wave events, creating seven scientific visualizations and two tables from over ten papers.

New UI features in Kimi Work—widgets and dashboards—allow users to generate interactive components directly in conversations and organize them into persistent, theme‑based views.

Architecturally, Kimi K3 builds on Kimi Delta Attention (a mixed linear attention mechanism) and Attention Residuals, enabling smoother information flow across longer sequences and deeper layers. The model expands Mixture‑of‑Experts sparsity to 896 experts, activating 16 per token, and introduces Quantile Balancing for routing, Per‑Head Muon optimization, SiTU activation, and Gated MLA to support stable training at the 2.8 T‑parameter scale.

Training employs quantization‑aware techniques with MXFP4 weights and MXFP8 activations. Expert‑parallel training uses static shapes without host synchronization, and inference benefits from a large high‑bandwidth communication domain. Deployment is recommended on supernodes of 64 or more accelerators, where the prefill‑cache‑enabled KDA delivers competitive token pricing even with long contexts.

Limitations include sensitivity to historical context; incomplete history return or model switching can cause context interference and unstable output quality. The model may make unexpected decisions when user intent is ambiguous. The authors advise using compatible agent frameworks (e.g., Kimi Code) and adding stricter constraints in system prompts to mitigate these issues.

Kimi K3 is available through kimi.com, the latest Kimi mobile app, the Kimi Work desktop client, Kimi Code, and the Kimi API.