RAX: Robot Agents Self-Improve via Online Exploration, Lead SOTA by 37.4%

From Scaling Data to Scaling Exploration

Robotics has long relied on scaling datasets and model sizes (VLA, WAM) to push capability limits. Yet zero-shot deployment on new robots or environments still demands massive human teleoperation data and policy retraining. The RAX (Robot Agent eXploration) framework, developed by researchers from Harvard, NVIDIA, UC Berkeley, MIT, and Stanford, shifts the paradigm: instead of scaling offline data, it scales online exploration at test time. A general-purpose coding agent connects directly to a robot's perception (RGB, depth, camera intrinsics, force feedback) and control interfaces. The agent writes perception and planning code, executes it, observes outcomes, and iteratively revises the program until the task succeeds. Successful programs are saved as reusable skills, forming a growing library. Crucially, the underlying model parameters remain frozen; capability grows solely through accumulated exploration experience.

RAX Architecture: Frontend Exploration, Backend Execution

RAX adopts a frontend-backend separation. The frontend handles exploration, program generation, memory, and skill organization. The backend provides low-level perception, control, and execution for a specific robot embodiment. The same agent framework connects to single-arm, dual-arm, dexterous hands, mobile bases, and humanoid robots. When a new robot is added, developers only implement the backend interface; the agent then learns tasks through interaction without any demonstration collection or policy training.

Benchmark Results: Large Gains with Zero Demonstrations

RAX was evaluated on five simulation benchmarks, averaging a 37.4 percentage-point lead over the previous best methods:

RoboDojo (hard subset): from 24.9% to 78.3%

BEHAVIOR-1K (hard subset): from 8.2% to 46.5%

DexJoCo (bimanual tasks): from 48.8% to 94.3%

LIBERO (original four suites): 98.5% average success with 0 demonstrations (vs. 50–100 demos per task for VLA/WAM)

LIBERO-Pro (position randomization, ≤5 adaptation episodes): 84.4% vs. VPP2 42.3% and ASPIRE 63.2%

On several tasks, all prior policy models scored 0%: RoboDojo language-instructed pouring (RAX 100%), Swap blocks (Physical RSI 0%, RAX 100%), and BEHAVIOR-1K long-horizon truck loading (10-minute task, all prior methods 0%, RAX 100%).

Exploration Scaling Laws

Performance improves predictably with more exploration episodes:

Charger insertion: 26.0% (4 episodes) → 94.0% (8 episodes)

Block swapping: 34.0% (4 episodes) → 100% (16 episodes)

This demonstrates a clear test-time scaling law : more exploration directly converts to higher success rates, and the gained capabilities are retained as executable programs.

Real-Robot Experiments: Precision and Long-Horizon Tasks

On physical robots (including the RB-Y1 mobile manipulator), RAX learned complex skills with zero human demonstrations and under 2 hours of autonomous exploration :

0.1 mm peg-in-hole: Transparent plate defeated depth sensing; agent used force feedback to measure height and correct pose.

Air fryer operation: Agent built geometry from depth, estimated handle orientation, discovered a 60° tilt release to fit a 19 cm banana into a 19 cm basket, and chose a shorter handle grasp to avoid inverse-kinematics limits.

Wiping, object tracking, irregular block stacking.

Long-horizon mobile whole-body control: Six-stage task (move chair, stack cups, place cup+banana in basket, move basket to cabinet, push chair back, transport box) composed from previously learned skills.

Inference latency dropped dramatically: the air fryer task required ~72 minutes during initial exploration but only 18 seconds when running the final program—a 241× speedup .

Code as Policy and Skill Reuse

Each successful exploration yields a standalone program ( Code as Policy ) that re-localizes objects, generates actions, and verifies critical steps (grasp confirmed? drawer open? object placed? retry needed?). Programs handle routine execution; when perception or planning fails, exceptions return control to the agent for debugging and program revision. Learned skills are composable: wiping code reused for tissue manipulation; transport, grasp, and chair-moving skills combined into the long-horizon sequence.

Fully Open Source

The team released the complete stack:

Paper: Scaling Online Exploration for Robot Agents

Project page & experiment sessions: https://robot-agent-exploration.com/ ArXiv: https://arxiv.org/abs/2610.22496 Framework code: https://github.com/robot-agent-exploration/rax Skill library: https://github.com/robot-agent-exploration/skills-lib Full experiment logs:

https://huggingface.co/datasets/robot-agent-exploration/robot-session-logs

Researchers can now connect new robots, launch exploration, inspect generated programs, and save successful skills for future tasks—establishing a scalable platform for embodied self-improvement (RSI).