RoboCoach: World Models Drive Efficient Active Error Correction in Embodied RSI

Introduction: The Challenge of Embodied Recursive Self-Improvement

Recursive Self-Improvement (RSI) focuses on whether a system can continuously discover its own shortcomings, acquire new experience, and use that experience to improve the next round of performance. Current embodied systems remain heavily dependent on real-robot trial-and-error and human decision-making. As tasks grow longer and skill combinations richer — while real-robot data collection stays expensive — the core question becomes: how to allocate limited human intervention where it matters most .

RoboCoach addresses this by training a world model to locate weak sub-task policies in long-horizon tasks, then guiding humans to provide targeted demonstrations that update the corresponding policy experts. This achieves maximal capability improvement for the whole task with minimal intervention.

On October 5, 2024, Stanford teams led by Shuran Song and Chelsea Finn released Mulligan , which reached the same conclusion: "uniform collection is an expensive waste." The difference: Mulligan directs budget toward observed failure states, while RoboCoach directs it toward sub-tasks identified as weak by the world model.

Key Insights

Embodied RSI efficiency means maximizing effective iteration under a fixed budget. RoboCoach raised Franka success from 13.3% to 75.0% and AgileX from 40.0% to 83.8% under the same demonstration budget, far outperforming uniform-collection baselines.

Data value depends on complementarity with the current embodied foundation model. Targeted demonstrations informed by world-model diagnosis outperformed uniform demonstrations across four platforms (simulation and real).

World models learn transferable dynamics while action policies must balance real-time execution and task specificity. The world model substitutes for some real-robot validation in closed-loop iteration.

Compositional generalization of long-horizon tasks can be achieved efficiently by correcting sub-task policies. On four held-out tasks (reordering, cross-task stitching, and extension), RoboCoach achieved 35.0% average success versus 0% for a shared-policy baseline.

RoboCoach Design: Shared Prediction, Expert Execution

CoachWorld: A Unified Action-Conditioned World Model

CoachWorld receives historical visual observations, task text, robot actions, and camera geometry, then predicts subsequent visual changes. End-effector motion, gripper open/close, object contact and position changes are shared prediction targets across tasks.

To cover both single-arm and dual-arm robots, CoachWorld uses a unified dual-slot end-effector representation : each slot records 3D position, 6D rotation, gripper state, and a validity flag. Single-arm robots use one slot; dual-arm robots use two, unifying the action space.

Camera geometry aligns robot-frame actions with each viewpoint's image: the same 3D end-effector trajectory projects to different 2D directions and distances depending on the camera. Projecting the 3D trajectory into each camera and constraining video generation with trajectory and camera info gives actions a visual grounding.

This design lets data from different embodiments enter a shared prediction space. CoachWorld is initialized from Wan2.2 TI2V-5B and trained on ~500 hours of interaction data, including successes, failures, free exploration, and sub-optimal trajectories.

Modular Policy Experts: Balancing Real-Time Execution and Task Specificity

The world model predicts action consequences; the execution policy must output the next action under the current embodiment, task stage, and control timing. The former needs broad sharing of interaction knowledge; the latter needs locally adjustable parameters and context interfaces.

RoboCoach therefore organizes independently adaptable skill experts on top of a shared VLA backbone . In experiments, LIBERO and Franka used a π₀.₅ backbone; RoboTwin and AgileX used a MolmoAct2 backbone. Each expert is a separate LoRA adapter that carries the learnable capability. Each coaching round updates only the selected expert; other experts and the shared backbone stay frozen.

A "skill expert" is not a brand-new policy for every instruction. For example, "press red button" and "press green button" invoke the same pressing expert . The sub-task–expert pair maps the concrete operation (what to do, to which object) to a reusable module: the sub-task specifies the goal, the expert executes and receives new demonstrations for that behavior.

Modularity applies to the policy's adaptation and update points, not to fragmenting the same physical interaction skill by task name. One improvement in pressing capability propagates to different buttons and task sequences.

RIDI Loop: How the Robot Gets a Concrete "Remedial Lesson Plan"

RoboCoach structures the coaching process as Route → Imagine → Diagnose → Improve (RIDI) .

Route : The router converts the long-horizon instruction into an executable sub-task sequence based on the task instruction, current visual state, existing skill vocabulary, and completed prefix, then dispatches each sub-task to its expert.

Imagine : The current skill expert outputs actions given observations; actions are converted to end-effector trajectories; CoachWorld predicts the resulting video; the generated frames feed back to the policy for the next action. This forms a policy-in-the-loop closed-loop imagination rollout .

Diagnose : A progress discriminator evaluates each predicted video against the current sub-task and visual sequence, outputting a completion score. When the score crosses a threshold, the system advances to the next sub-task. If the time limit expires without completion, the first timed-out sub-task and its expert are recorded.

Improve : First-timeout records across tasks are aggregated into a scorecard showing overall imagination success rate, per-sub-task reach/failure statistics, progress curves, and representative failure videos. Human collectors review the failure videos, perform safety checks, and provide demonstrations.

After targeted remediation, only the grasping expert is updated; placing and pouring experts remain unchanged. Both real execution and CoachWorld imagination then complete the full sequence: grasp tea bag → place in cup → pour water.

Module Validation

1. World Model: CoachWorld Predicts Long-Horizon Action Execution

On held-out test sets, all models receive identical history images and future action conditions. Metrics compare visual fidelity, end-effector trajectory following, and human-rated physical plausibility and instruction following.

On DROID-180 , CoachWorld (mixed-domain training) leads in 6 of 7 major metrics: LPIPS 0.0996 , FVD 51.71 , three end-effector trajectory metrics, and instruction-following score.

On Lab Franka-180 , LPIPS 0.1712 (lowest among compared methods).

The project page shows action-control examples and long-horizon real-vs-prediction side-by-sides. The longest closed-loop example reaches 85.8 seconds : the model generates video segments iteratively, feeding predictions back to the policy.

To measure how well imagination reflects real deployment, the team evaluated full-task success rate across 22 task–policy combinations in LIBERO, RoboTwin, Franka, and AgileX. Imagination success rate correlated with real success rate with Spearman ρ = 0.840 and Pearson r = 0.820 , indicating the world model reliably ranks policy performance differences. RoboCoach can therefore screen tasks and behaviors for priority before costly real-robot coaching.

2. Progress Discriminator: Sub-Task Completion Signals

RIDI needs to decide when to switch experts. Switching too early (before grasp is stable) or too late (after grasp is done) both hurt performance. The team compared six discriminator methods under a unified causal-observation protocol, evaluating the binary decision "is the current sub-task complete, should we switch?" using F1.

RoboMeter — a visual-trajectory-based real-time progress estimator — was adopted as RoboCoach's progress discriminator and outperformed alternatives including LIV, Contrastive λ, TOPReward, and SuccessVQA.

Three Coaching Rounds, Limited Demonstrations: How Much Does Full-Task Success Improve?

Four Evaluation Domains, Unified Demonstration Budget

Simulation : LIBERO-Long (10 long-horizon tasks) and RoboTwin 2.0 (5 dual-arm tasks) covering object placement, drawer/switch operation, stove manipulation, block stacking, bowl stacking.

Real robots : Franka Research 3 (3 tasks) and AgileX dual-arm (4 tasks) covering switch/lamp cord, continuous button pressing, bread placement with lid, block into drawer, shoe boxing, table wiping with plate arrangement, and tea making — all with clear sequential dependencies.

All methods start from the same initial policy and run three update rounds under a fixed budget. Each round adds 100 qualified sub-task demonstrations in simulation and 50 per real-robot platform . The comparison is about how the same amount of new data is allocated and used, not about simply adding more data.

Full Results Across Four Domains

Franka : RoboCoach: 13.3% → 46.7% → 68.3% → 75.0% ; Uniform baseline: 13.3% → 15.0% → 20.0% → 30.0% .

AgileX : RoboCoach: 40.0% → 72.5% → 83.8% → 83.8% ; Uniform baseline: 40.0% → 48.8% → 46.3% → 47.5% .

Per-Task Policy Iteration Details (Real Robot, Round 3)

Platform averages hide task-level dynamics. Example improvements after three rounds:

Tea making : initial 10% → RoboCoach 90% vs. uniform baseline 20%.

Table setting : initial 65% → RoboCoach 100%.

Enhanced Skills Exhibit Stronger Compositional Generalization

If each skill only works on one fixed trajectory, new instructions still require new long-horizon demonstrations. The appeal of modular learning is that local reinforcement can be reused in new task compositions .

The team constructed four unseen full-task sequences on two real robots, testing long-horizon extension, cross-task stitching, and skill reordering. The four new sequences achieved 35.0% average success ; the uniform-collection baseline with a shared adapter scored 0% . Notably, "set table then make tea" (stitching two learned sequences) succeeded 13 out of 20 real-robot trials.

This validates the design principle: skills should be reusable capability units that continue to produce value in new tasks . For long-horizon tasks, improving one weak skill benefits not just the training trajectory but all future task combinations requiring that behavior.

Outlook: From Active Coaching Toward Fully Autonomous Embodied RSI

This work answers "how can embodied systems efficiently and actively correct errors?" Mulligan turns "from which failure state to start" into an optimizable variable, concentrating collection where the policy actually fails. RoboCoach turns "which sub-skill failed" into a queryable object, routing each demonstration to the correct expert. When a long-horizon task chains multiple sub-skills, pinpointing the exact sub-task and expert is far more data-efficient than blindly retraining the whole sequence.

The next step for embodied RSI is a more autonomous loop: a world model with strong physical reasoning uses expected failure states to decide where to collect , progress monitoring diagnoses what to collect and which expert to update , and humans only need to provide a single demonstration at the most critical step .

Paper and Resources

Paper: Jiajun Liu, Yifan Chen, Yichao Liu, Jiayi Zhang, Ruoqu Chen, Shaoxuan Xie, Guocai Yao, Mengdi Xu, Sen Cui, Changshui Zhang. RoboCoach: World Models as Active Coaches for Compositional Robot Skills. arXiv:2609.39685 (2026).

Paper PDF: https://arxiv.org/abs/2609.39685

Project page & videos: https://robocoach-ai.github.io/

CoachWorld open-source code: https://github.com/RoboCoach-AI/CoachWorld

Pre-trained weights: https://huggingface.co/JEdward/CoachWorld

Related work: Mulligan (Lars Ankile et al., arXiv:2610.05882, 2026).