3B Activation Parameters Enable State‑of‑the‑Art Agentic Coding: KAT‑Coder‑V2.5‑Dev Open‑Source Release

In July the KwaiKAT team released KAT‑Coder‑V2.5, which ranked first on PinchBench for agent tool calls and second on SWE‑Bench Pro only behind Opus 4.8. The open‑source Dev version follows the same design.

The model uses an MOE architecture with 350 B total parameters and 3 B activation parameters, and adopts Qwen3.6‑35B‑A3B as the base model for post‑training, making it runnable on consumer‑grade GPUs.

Training proceeds in two steps: supervised fine‑tuning (SFT) on 127 K samples, then reinforcement learning (RL) on the SFT model. The team stresses that the bottleneck for agentic coding lies in training infrastructure rather than model size.

The AutoBuilder engine reconstructs real repositories into reproducible, executable sandbox environments. It extracts tasks from pull requests and commits, obtains golden and test patches, and generates structured task descriptions (problem statement, behavior requirement, interface constraints). Ambiguous samples are filtered out. A build agent creates configuration scripts, installs dependencies, and runs tests; a verification agent executes the scripts in isolation and validates results by parsing structured test output. An environment is considered successful only if over 90 % of expected tests are collected and reproducible across multiple runs. Success rates rose from 16.5 % to 57.2 %, yielding more than 100 k verified environments covering 12 programming languages.

Data scaling follows a “flywheel” approach. Many near‑miss trajectories are rescued through a two‑stage hint‑in‑the‑loop process: first, process‑level hints raise the pass rate to ~20 %; second, validated patches are used to regenerate unprompted trajectories that retain only valid, non‑leaked samples. Rule‑based gating and heuristic scoring filter out invalid, unstable, or exploitative trajectories, scoring dimensions such as exploration, localization, edit reasoning, spec fidelity, repository conventions, patch minimality, verification quality, recovery behavior, and honesty. Harness randomization (tool name, parameter format, output format, injected missing dependencies, transient failures, noisy logs) prevents overfitting to a single execution framework.

The system introduces KwaiClawEnv, which consists of Service, Task, and Eval layers. The Service layer parses open‑source Skill definitions (e.g., OpenClaw) achieving >90 % success, and augments under‑covered domains with LLM‑generated variants. The Task layer generates millions of candidate tasks, filters them through multi‑stage verification, and retains over 100 k high‑quality instances, with an average of 15 tool calls per trajectory and some exceeding 100 steps.

Four key RL design elements are retained: (1) Token‑in‑Token‑out (TITO) consistency—bypassing chat interfaces and calling the /generate endpoint directly to avoid retokenization drift observed in ~40 % of ~200‑round tasks; (2) Truncated Importance Sampling (TIS) to cap large importance weights and control variance; (3) Reliable sandbox and validator—early crashes traced to sandbox failures (≈16 % of trajectories) rather than policy issues; (4) Hierarchical, harness‑based rewards comprising Core Task Score (full credit only if all fail‑to‑pass and pass‑to‑pass tests succeed), Standard Behavior Constraints (penalties for content duplication, garbled output, incorrect tool usage, etc.), and Failed Trajectory Incentives (positive reward for meaningful progress in failing trajectories). Additionally, a judge model GRM scores trajectories on fault diagnosis, post‑repair verification, and execution strategy using a rubric and RL fine‑tuning with human‑labeled data.

Stability improvements include redesigning image management to lower peak disk usage from 95 % to 60 % and reduce invalid rollouts from 6‑7 % to <1 %; fixing environment‑variable overrides cut reward noise from ~6‑7 % to <1 %; overall sandbox error rate dropped from ~16 % to <2 %, and training crashes decreased by an order of magnitude.

A Qwen3.6‑specific issue arose: a binary 0‑1 reward caused collapse after the second epoch, and the model began issuing excessive parallel tool calls (sometimes >70 per round), inflating context length and generating many invalid trajectories. The team added Qwen3.6‑specific penalties for over‑parallel calls, failed calls, empty call blocks, and duplicate content, which stabilized training for 10 epochs with continuously rising rewards.

Ultimately, the Dev version achieves state‑of‑the‑art agentic coding performance at comparable scale (350 B total, 3 B activation) on consumer hardware. All V2.5 post‑training components—including environment construction, data scaling flywheel, harness randomization, sandbox hardening, asymmetric PPO, hierarchical rewards, and multi‑teacher distillation—are released, along with the Qwen3.6‑specific pitfalls and reward adjustments, providing a comprehensive reference for the open‑source community.