ClawLoop: Harness-Native Agent Self-Improvement Loop Without Stronger Teacher Models

ClawLoop Framework Overview

Meituan and Peking University introduce ClawLoop , a Harness-native agent self-improvement training framework. The term "Harness-native" means task generation, training trajectory collection, and diagnostic evaluation all operate within the same execution environment where the agent manipulates files, runs commands, uses tools, and has its results checked by verification mechanisms.

ClawLoop connects three phases into a closed loop:

Phase 1: The agent generates and filters tasks, collecting execution trajectories.

Phase 2: The system uses these trajectories for supervised fine-tuning (SFT) and reinforcement learning (GRPO) to update the model.

Phase 3: The updated agent completes held-out evaluation tasks; failures are analyzed to inform the next round of task generation.

Each updated model re-participates in the next round, forming a self-improvement cycle without requiring a stronger external teacher model.

Task Generation: Two Complementary Paths

Task quality is critical; ClawLoop employs two complementary generation paths:

Capability-driven: Starts from capabilities (e.g., cross-source verification, multi-document merging) and instantiates them across different domains.

Domain-driven: Starts from professional domains (e.g., database management) and combines domain tools and skills to generate exercises.

Both paths together cover general capabilities and domain knowledge. Ablation experiments show that keeping only one path degrades overall performance, confirming their complementarity.

Verification and Difficulty Filtering

Generated tasks undergo verification:

Reference solutions must run and pass the verification program.

Checks for answer leakage and trivial tasks (where no action yields a score).

The current agent attempts each task multiple times; the main experiment collects 8 trajectories per task.

For reinforcement learning, tasks with success counts between 1 and 5 (out of 8) are retained.

The rationale: tasks always solved provide limited new learning signal; tasks never solved may be too hard or flawed. The goal is to continuously supply tasks the agent has a chance to learn but has not yet mastered.

Training: SFT + GRPO

Validated tasks and their trajectories feed two training objectives:

Supervised Fine-Tuning (SFT): Uses successful trajectories from the agent itself, teaching verified effective behaviors.

Reinforcement Learning (GRPO): Uses task completion outcomes as rewards to further optimize execution policies.

Thus, "self-improvement" in ClawLoop includes actual model parameter updates. Diagnostic feedback influences subsequent training data; training converts execution experience into changed model capabilities.

Diagnostic Feedback: From Failure to Next-Round Tasks

After each round, the agent analyzes its own failure trajectories on held-out tasks, recording three aspects:

At which step the failure began.

The root cause.

Which capability is missing.

The system aggregates these records by skill category to find recurring failure patterns. For example, in a multi-file refactoring task, the agent modified the target file but failed to update import statements in dependent modules. When this pattern repeats, the next round generates tasks targeting dependency-handling capability.

A key design distinction: diagnostic reports guide task generation; task completion is still judged by the verification mechanism. The agent's self-explanation is not used directly as a reward signal, separating self-analysis from outcome verification.

Experimental Results

Base model: Qwen3.6-27B . Main experiment runs 3 rounds , each generating 1,024 new training tasks . Evaluation covers three benchmarks:

SkillsBench (skill usage): 46.3 → 57.2 (+10.9 points)

Claw-Eval-Live (real-world scenarios): 69.3 → 76.9 (+7.6 points)

ClawGym-Bench (executable tasks): 67.6 → 68.4 (+0.8 points)

All three improve, but gains are uneven; SkillsBench benefits most, ClawGym-Bench least.

Cross-Model Transfer

Generated tasks were also used to train smaller Qwen models:

Qwen3.5-4B: Claw-Eval-Live 32.4 → 48.0

Qwen3.5-9B: Claw-Eval-Live 58.3 → 71.7

This demonstrates cross-model benefit of the generated curriculum.

Plateau Analysis: Difficulty and Iteration Limits

The study observes diminishing returns:

Task difficulty: Harder tasks produce longer trajectories, but longer trajectories do not guarantee better training gains. Moderate difficulty balances challenge and success rate.

Iteration count: SkillsBench continues to benefit from more rounds, but Claw-Eval-Live and ClawGym-Bench plateau around 3 rounds .

These findings highlight open questions in recursive self-improvement (RSI): whether feedback yields improvement, how long improvement persists, and where bottlenecks appear.

Context: RSIHub and Related Work

The article notes concurrent work like RSIHub (ByteDance), which organizes execution, analysis, modification, and evaluation into composable pipelines for agent evolution experiments. ClawLoop differs by embedding task generation, training, and diagnosis within a single Harness environment, letting execution experience directly feed the next training round.

ClawLoop provides a concrete path for agent self-improvement: model completes tasks → system verifies results → new tasks and training drive the next update. It demonstrates measurable gains in limited iterations, while identifying future challenges: sustaining valid task generation, improving diagnosis quality, and breaking through iteration plateaus.