How End‑to‑End Reinforcement Learning Powers the Kimi‑Researcher AI Agent

Background and Objective

Kimi‑Researcher is an AI agent designed to perform autonomous research tasks rather than simple retrieval. The system is trained from scratch using end‑to‑end reinforcement learning (RL), enabling the model to evolve its reasoning capabilities without human‑crafted pipelines.

Design Goals

Enable long‑term, multi‑step “thinking” in the agent.

Apply end‑to‑end RL so the model can improve autonomously.

Limitations of Conventional Agent Approaches

Workflow assembly : Manual prompt‑based pipelines that combine multiple agents, planners, and sub‑tasks. Changing the underlying LLM requires redesign of the whole workflow and often depends on services unavailable in certain regions.

Supervised fine‑tuning (SFT) : Collecting human‑annotated task trajectories for imitation learning. This approach is labor‑intensive and does not scale to large data volumes.

Advantages of End‑to‑End Reinforcement Learning

Dynamic action generation

In an RL setting the agent receives a reward signal for task success and generates actions on‑the‑fly, allowing it to tackle novel problems without redesigning a fixed pipeline.

Data‑ and compute‑driven scaling

When performance on a problem class drops, the same problem instances are added to the training set and additional compute is allocated. The performance ceiling is therefore limited by available data and compute rather than human design.

Scalable on‑policy data collection

The agent continuously explores a simulated research environment, producing on‑policy trajectories as long as a reliable reward function can be defined. Increasing rollout volume directly yields more high‑quality training data.

Empirical Results

On the “Humanity’s Last Exam” (HLE) benchmark the model’s score rose from 8.6 % to 26.9 %, a gain attributed primarily to RL training. This places Kimi‑Researcher among the top global systems, comparable to the improvement reported by OpenAI’s Deep Research team (≈20 % → 26.6 %).

On the HLE test set the agent achieved a pass@4 of 40.17 %, meaning it solves more than four‑tenths of difficult problems within four autonomous attempts.

Observed emergent behaviours

After producing an initial answer the agent performs additional search rounds, cross‑validating information from multiple sources before finalising the result.

When confronted with a highly specialized question lacking existing information, the agent proposes actions such as contacting the original paper’s author; such actions are intercepted for safety but demonstrate goal‑directed planning.

Representative Use Cases

Benchmark discovery

Prompt:

Survey all advanced benchmarks that frontier LLM scores lower than 20%, focus on text. example like HLE

The agent identified previously unknown benchmarks including AGI‑2, FrontierMath, and Seal QA.

Knowledge‑structure mapping

Prompt:

Analyze the evolution of three major monetary systems: gold standard, Bretton Woods, floating‑rate regime

The output was a timeline‑based, structured overview suitable for teaching or literature review.

Rapid domain overviews

Prompt:

List data‑privacy laws of Southeast Asian countries, provide brief summaries and key takeaways for each

The agent generated a multi‑thousand‑word report covering ten jurisdictions, with comparative tables of provisions.

Domain‑specific analysis (fictional example)

Prompt:

Research the skill panels of main players in the manga “Slam Dunk” and produce a scouting report

The agent produced a detailed player‑analysis report.

Complex product recommendation

Prompt:

Explain price differences among portable juicers with similar features, identify real versus hype features, and recommend reliable models within a 100‑yuan budget

The response included a breakdown of functional claims, cost drivers, and a shortlist of affordable, trustworthy models.

Conclusion

End‑to‑end reinforcement learning enables an AI agent to acquire research‑level capabilities, exhibit emergent problem‑solving behaviours, and scale with data and compute far beyond traditional workflow‑based or supervised‑fine‑tuning methods.