DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Human Whole-Body Demos

Complex mobile manipulation tasks — such as lifting a fruit basket, turning, and placing it on another table — require continuous coordination of a mobile base, torso, dual arms, head, and dexterous hands. Collecting high-quality robot teleoperation demonstrations for such whole-body skills is costly and often requires multiple operators. The authors ask whether the rich whole-body coordination experience already present in everyday human activity can be transferred directly to robots.

System Overview: DexRoam

DexRoam is an end-to-end framework that collects egocentric whole-body human demonstrations, aligns them to a target robot embodiment, and uses the aligned data jointly with robot data to train vision-language-action (VLA) policies. The work was accepted at CoRL 2026.

Lightweight Data Collection

Human demonstration collection: A portable system uses only a Meta Quest 3 headset and a head-mounted ZED Mini stereo camera. The demonstrator wears both devices; the system synchronously records first-person video, full-body joint motion, and hand keypoints without external motion capture, environment markers, or additional body trackers. The camera feed is streamed live into the headset so the visual experience during collection matches the policy's observation space.

Robot demonstration collection: A single-operator whole-body teleoperation system combines a VR headset, controllers, and data gloves to simultaneously control the robot's base, torso, dual arms, head, and dexterous hands. Both systems together provide the paired human and robot datasets needed for joint training.

Three-Stage Human-Robot Action Alignment

To bridge embodiment gaps while preserving continuous whole-body coordination, DexRoam applies three sequential alignment stages:

Embodiment alignment: Using the human pelvis as a reference frame, the method maps human base, torso, dual-arm, and head motions to the corresponding robot links. Hand retargeting generates dexterous-hand commands, preserving relative motion relationships across the whole body.

Semantic alignment: Residual differences in body proportions (height, arm length) and initial poses are reduced by representing base, torso, and arm motions as relative action representations, making the control semantics consistent between human and robot demonstrations.

Temporal alignment: Human and robot execution speeds differ. DexRoam resamples human trajectories according to task progress, matching the robot's time scale while retaining the original motion shape, yielding temporally consistent supervision.

After alignment, human and robot demonstrations share a unified observation-action interface and can be fed directly into VLA policy training without architectural changes.

Policy Training

The team evaluates two VLA backbones — GR00T N1.7 and π₀.₅ — under three training strategies: (1) joint human-robot training, (2) human pre-training followed by robot fine-tuning, and (3) human pre-training followed by joint human-robot fine-tuning.

Experimental Results

Alignment Quality

PCA visualizations of action distributions show clear separation after embodiment alignment alone; after the full three-stage pipeline the distributions overlap substantially. Quantitatively, Maximum Mean Discrepancy (MMD) and Sliced Wasserstein Distance (SWD) drop by 41.7% and 31.1% on average, respectively. Aligned human trajectories can be successfully replayed on the real robot, confirming executability.

Real-Robot Policy Performance

Across five tasks and both VLA models, all three human-data-augmented strategies improve over robot-data-only baselines. The best strategy — human pre-training + joint human-robot fine-tuning — raises average success rates from 29% to 56% for GR00T N1.7 and from 32% to 57% for π₀.₅ .

Human demonstrations also introduce novel behaviors absent from robot data. In a chip-can pickup task, the robot-only policy consistently grasps from the front and often knocks the can over when its orientation changes. With human data, the policy learns to switch to a side grasp depending on object orientation — a strategy present only in human demonstrations.

Reducing Robot Data Requirements

Holding the amount of human data constant while halving the robot demonstrations yields task performance nearly matching the full-robot-data baseline on both VLA models. This indicates that aligned human demonstrations can substitute for roughly 50% of high-cost whole-body teleoperation data . Few-shot experiments further show that even a small amount of robot data combined with human data significantly boosts performance.

Conclusion and Open-Source Release

DexRoam demonstrates that human whole-body demonstrations can cross embodiment gaps to serve as direct action supervision for mobile bimanual dexterous manipulation, enabling robots to learn continuous whole-body coordination rather than isolated grasping or navigation skills. The authors argue that the value of human data lies not only in visual and task diversity but in the fine-grained, interlinked whole-body motion experience it contains. As this experience becomes transferable, complex manipulation learning can reduce its heavy dependence on expensive whole-body teleoperation datasets.

The team has open-sourced the collection and alignment code ( https://github.com/zhourui9813/DexRoam) and the policy training code ( https://github.com/zhourui9813/DexRoam-Policy-Training), along with the project page ( https://dexroam.github.io/) and arXiv preprint ( https://arxiv.org/abs/2609.35761).