Ropedia Xperience 2: Gold Standard for Scaling Physical AI Environments and Behaviors

Introduction: The Need for Embodied Data in the Post-GPT-6 Era

General large models are rapidly improving in embodied capabilities. Anthropic's Claude Fable 5.2 scored 64.9% on the RoboDojo robot control benchmark, more than double its predecessor (27.6%). Two months earlier, GPT-6 Astra topped the same leaderboard with a 22.48% average success rate across 40 public policies. However, benchmark scores do not equal real-world capability. A joint study deployed GPT-6 Astra on eight real robot configurations: it failed all 11 attempts at plug insertion and succeeded in only 9 of 23 long-horizon navigation-and-retrieval trials. Researchers concluded the model has preliminary embodied skills but remains far from practical deployment.

To bridge this gap, models need both behavioral demonstrations for learning and environments for executing tasks and accumulating experience. While large models follow scaling laws, the scaling law for physical intelligence requires clarity on what environment and behavior data to scale, and how to continuously produce and use it. Physical AI startup Ropedia answers with Xperience 2.

Xperience 2: Real-World Scenes Aligned with Human Demonstrations

Xperience 2 is a dataset of real environments and behaviors for Physical AI. It fuses real-world scene scans with first-person human activity recordings, aligning scene geometry, video, trajectories, and body poses in a single metric coordinate system. From this aligned data, Ropedia converts human motions into robot trajectories and scanned scenes into interactive simulation environments. The goal: establish a "gold standard" for next-generation embodied data, providing the data foundation for scaling laws of physical intelligence in the post-GPT-6 era.

Scan, Record, and Align: Giving Every Behavior a Real Environment Reference

Each task recording retains its corresponding scene reconstruction. Camera trajectories, hand and full-body motions are placed in the same 3D coordinate system, aligned with room geometry and layout. Users can replay how a person acted in the room and reuse these records to construct robot tasks.

Example: placing a cup in a cabinet. Video captures the sequence of picking up the cup, opening the door, placing the cup, and closing the door, providing action and order for model learning. But the robot also needs to know where the cup is, which way the cabinet door opens, whether its arm can reach, and how to judge task completion — information requiring scene geometry and object state. Video alone gives no environment for the robot to practice.

Conversely, simulation scenes alone are insufficient. Without human demonstrations, the robot must explore grasp positions, door opening directions, and placement actions from scratch, requiring reward design. As task steps increase, training cost grows.

Existing datasets typically focus on one aspect: ScanNet++, Realsee3D, DL3DV-10K provide 3D scene reconstructions and camera poses but no first-person task videos; EPIC Fields, Nymeria, HD-EPIC record human activities and camera poses but rarely provide scene meshes. Xperience 2 unifies both. After alignment, human motions have explicit spatial references, can be retargeted to robot morphologies, and validated in the same scene. Original videos remain usable for robot motion research.

Once a scene is configured for interaction, object positions, door states, and robot start poses can be set, letting the robot execute tasks. Failures (missed grasp, cup on edge, door left open) are detected via object state and recorded. The scene can be reloaded for another trial; new initial conditions yield new trajectories.

Data Layers in a Unified Metric Coordinate System

The aligned data includes: mesh, 3D Gaussian Splatting (3DGS), first-person video, head trajectory, dual-hand and full-body poses, and three timescales of activity annotations — all in a gravity-aligned coordinate system.

Two-Step Collection with Portable Devices

Collection continues Ropedia's first-person data expertise. Xperience-10M contained ~10M real-world interaction clips and 10,000 hours of egocentric video/audio. Xperience 2 adds scene scanning and spatial alignment, giving every activity a corresponding real environment. One scan serves multiple task recordings in that scene.

Two devices are used: a handheld LiDAR scanner captures the room from multiple angles, covering areas the human view misses. For task recording, operators wear Ropedia's HOMIE Headset 2, freeing both hands. Its four global-shutter fisheye cameras, IMU, and four-channel audio record activity; two forward cameras capture hands and manipulated objects. The portable setup enters homes, restaurants, shops, offices, and outdoors.

Process: operators first scan the room, obtaining a metric-scale scene mesh, 3DGS, and corresponding images, camera poses, and sparse point cloud — all in a gravity-aligned coordinate system. After adding physical properties and interactive parts to objects, the scene becomes simulatable.

Then operators don HOMIE Headset 2 and perform assigned tasks. Each recording spans from before task start to after completion, preserving a full activity.

Videos undergo automated processing to extract camera trajectories, depth, and estimated dual-hand and full-body poses. Video, IMU, and audio are time-aligned. Motions receive three-layer language annotations: overall task, subtask timing, and per-frame object and hand states.

Alignment: the system locates video frames in the scanned scene, transforms camera trajectory, hands, and body into the scene coordinate system. Registration checks validation frames, gravity direction, and depth-mesh consistency, labeling each recording PASS, WEAK, or FAIL for user filtering. Processed trajectories and poses are kept alongside raw video for verification.

From Bedrooms to Streets: Bringing Real-World Diversity into Data

The current release covers 1,589 scenes across 12 real-space categories. Residential interiors: 1,389 scenes (87.4%); bedrooms and living rooms combined: 976 (61.4%). Collection spans 596 locations, including 443 entire homes.

By whole-task count: 74 tasks, 4,230 recordings, covering pick-place, cleaning, organizing, folding, and inspection. By primary verb: 25 pick/place tasks, 23 wiping/cleaning, 18 arranging/organizing. 31 tasks originate from workplace and retail scenes (restaurants, shops, offices).

High-Precision Data: Verifying Models' Spatial Capabilities

Robots entering unfamiliar rooms must reconstruct 3D structure, locate objects, and understand distance and direction. Xperience 2 builds three benchmarks on the same 20 real scanned scenes, using scan geometry, camera poses, and spatial annotations as ground truth to concretely measure model capabilities in real environments.

Scene Reconstruction: Leading Models Show Promise

Recovering a room from images requires simultaneous surface and camera motion recovery. Among 11 models, π³ achieves lowest trajectory error and highest surface reconstruction F-score across three input densities; DA3-Giant leads in pose recovery for medium and dense inputs. With scan geometry and camera poses as reference, depth, trajectory, and surface reconstruction errors can be measured separately. More diverse real scenes enable broader training and evaluation across layouts and viewing conditions.

Object Localization: Real Environments Remain Challenging

After structure recovery, models must locate objects — identify targets and predict 3D bounding boxes. Best performer Boxer achieves [email protected] of 24.69% and [email protected] of 6.09%. Precise localization in real environments is still difficult, highlighting the need for more open real-scene data to cover diverse layouts, occlusions, and object variations with reliable 3D annotations.

Spatial Reasoning: Strong Performance on Spatial QA

The spatial reasoning benchmark contains 500 questions covering translation, rotation, distance, depth, height, and relative position. Gemini 3.8 Flash scores 74.2% overall; GPT-6 Astra (High) scores 71.8%. Both exceed 90% on height and relative position judgments, showing strong multi-type spatial question answering.

Next step: translate spatial understanding into action. Models must use position, direction, and object relation judgments for navigation, manipulation, and long-horizon task completion. Xperience 2 pairs real environments with human behaviors, providing data for learning "how to complete tasks in such spaces."

From Real Scenes to Virtual Training Grounds: Expanding Robot Experience Generation Paths

The same real scenes and human activities can produce different types of robot data. The technical report demonstrates four uses: Real-to-Sim converts scanned scenes into interactive simulation environments; Action Retargeting converts human demonstrations into robot motion references; Virtual Teleoperation collects new interactions in simulation; Long-Horizon Trajectory Generation uses programs to continuously execute multi-step tasks, producing full trajectories.

One real-world collection of scenes and recordings can be reused across these pipelines. New robot motions and interaction records accumulate beyond the original videos.

Real-to-Sim: Building Interactive Task Environments

Real-to-Sim uses scanned geometry and appearance to reconstruct individual objects, then adds collision, joints, and physical properties for simulated manipulation. In a real home demo, 107 objects were reconstructed, 73 made interactive.

The converted scene retains scan scale and coordinates, allowing pixel-wise comparison with scan reconstructions from the same camera view. In the illustrated view, median distance between simulated surfaces and scan mesh is ~2.5 cm; ~79% of pixels differ by ≤5 cm.

Control: GPT-6 Astra built a scene from collected images alone, without 3DGS. Object colors, sizes, and positions deviated; 3D bounding boxes of TV, fridge, microwave, and kettle shifted from their 3DGS positions. This control uses 3DGS as reference to compare two scene-building methods' consistency with the scan.

After independent object modeling, behaviors are configured per task. The 73 interactive objects include 9 articulated objects (fridge door, cabinet doors, drawers), 8 finite-element soft bodies (cushions), and 4 cloth pieces (TV cover, towels). Switches and remote buttons are pressable; faucets dispense water. Seven interaction types run in a single physics simulation.

Example: collecting a cup. Cup, cabinet body, and door are modeled separately, enabling door opening, cup placement, and door closing. The configured scene can be reloaded repeatedly, with tasks and start poses swapped. One real scan thus supports multiple robot experiment rounds.

Action Retargeting: Providing Robot Motion References from Human Demonstrations

Human motions already aligned to the scene are retargeted. A human turning while reaching may require a wheeled humanoid to simultaneously move base, rotate torso, adjust arm and dexterous hand. The system computes trajectories for each robot part under its kinematic and dynamic constraints, then replays them in the scanned scene. Thus one human task recording serves robot motion research without the robot on site.

Trajectories undergo velocity, torque, stability, and collision checks: joint velocity and torque ≤90% of limits, zero-moment point inside support polygon, no self or scene collisions. Head and fingertip tracking errors are logged.

Demo used 4 full recordings in the same scene: place package & operate AC remote, laundry, clear dishes, fold towels & sweep. Original duration 10.3 minutes. Videos slowed to 0.5× for retargeting; robot base traveled 96 m total, with 35–41% frames in motion per clip.

Fingertip tracking error medians: 0.40–0.55 cm; 95th percentile <2 cm. All frames pass constraints; peak joint velocity and torque at 90% and 89% of limits; zero-moment point ≥0.46 cm from support polygon boundary.

This yields robot motion references replayed in 3DGS where objects don't move. Whether the robot actually grasps the cup and places it stably in the cabinet requires verification in an interactive simulation.

Virtual Teleoperation: Collecting New Interaction Experience in the Environment

Virtual teleoperation connects an operator to the converted simulation. The human controls the robot in real time; objects react to actions. The system logs robot observations and control commands, checking command tracking, collisions, and joint limits. After a missed grasp, the operator adjusts based on current state, altering subsequent actions.

After each trial, reloading the scene resets it for the next demonstration. The same environment records multiple executions, different initial states, and failure recoveries. This still needs an operator but eliminates physical robot transport and manual reset.

Code-as-Policy Long-Horizon Trajectory Generation: Continuously Scaling Robot Experience

For long tasks, robots must chain subtasks: cup placed in microwave → retrieve from microwave → close door → continue. In a joint study led by SenseTime, tracking multiple subtask progress remained a GPT-6 Astra weakness.

Demo: an 8-step task in the Real-to-Sim home. Wheeled dual-arm robot: pick cup from kitchen counter, open microwave, place cup inside, close door to heat, retrieve cup, close door again, open bedroom door, place cup on bedside table. Robot navigates kitchen to bedroom, performing operations en route without scene reset.

Execution driven by a policy program using whole-body control. Program observes objects from perception only, not simulator ground truth. Changing robot and cup initial poses lets the same program generate full trajectories from different start states.

Across 30 reserved initial states (varying robot and cup positions), all 30 executions completed all 8 steps and passed task-rule checks. Verification uses simulator object states: cup must not tip or fall; appliances and doors must be in required states.

Each trajectory: 3,869–4,255 control steps at 30 Hz, averaging 2.2 minutes simulated time. Single simulation process generates one trajectory in 10–16 minutes. With scene and program ready, more trajectories can be generated by varying initial conditions and adding compute — no new human recordings needed.

These 30 trajectories share one scene, one task, one policy program, adding diverse full executions under different initial conditions. After scene, robot, and task rules are prepared, trajectory generation continues for downstream data production and model research. Environment coverage and behavior experience can be tracked and expanded separately.

Cost and Scale: Enabling More Environments to Carry More Behaviors

Real-robot collection demands sustained equipment, personnel, and time. DROID collected 350 hours of robot demonstrations in 564 scenes using 50 operators over 12 months. Robots must be transported; operators control throughout; manual reset after each trial.

Xperience 2 front-loads scene scanning, conversion, and task configuration. Once built, scenes are reloadable; recorded motions are retargetable; virtual teleoperation avoids physical robots and manual reset. One collection's scenes and behavior records serve multiple pipelines.

Long-horizon generation mainly consumes compute. At 10–16 minutes per trajectory, a single simulation process running 24 hours could theoretically produce 90–144 trajectories. Simulated and real-robot data fidelity differ; these paths complement real collection, with real-robot experiments needed for validation.

Collection locations are more flexible. Operators bring scanner and HOMIE Headset 2 on site — no robot transport needed. Multi-room scans preserve inter-room layouts, enabling robots to walk and manipulate across rooms in simulation. The 8-step task extended from kitchen to bedroom; generated tasks and trajectories need not be limited to original recordings.

From Xperience-10M to Xperience 2: Turning Data Scale into Physical Intelligence

From Xperience-10M (early 2026) to Xperience 2, Ropedia follows a clear path: first, record human behavior in the real world at scale; second, place those behaviors back into the real spaces where they occurred; third, make environments and behaviors together into data that robots can learn from, practice in, and verify against.

Large model progress rests on internet-scale text; physical intelligence progress similarly needs a sufficiently large, sufficiently real, continuously scalable data foundation. As general models break simulation records, the true bottleneck shifts to real-world environments and experience. Ropedia aims with Xperience 2 to establish scaling laws for physical intelligence's environments and behaviors in the post-GPT-6 era, making the question "how does data scale translate into model capability" measurable, verifiable, and ultimately answerable in the physical world.

References

[1] An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond

[2] Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies

[3] Has GPT-6-Astra Achieved Embodied Intelligence?

[4] Khazatsky et al. DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset. RSS 2024.

[5] Jiqizhixin: Physical AI Entering Experience Engineering Era, Ropedia Focuses on Full-Stack Data Infrastructure, Completes Tens of Millions USD Financing, Aug 6, 2026.

[6] Technical Report: Ropedia Xperience 2: A Gold Standard for Scaling Physical AI Environments and Behaviors — https://arxiv.org/abs/2611.17060 Project homepage: https://ropedia.com/xperience/2-research Dataset:

https://huggingface.co/datasets/ropedia-ai/Ropedia-Xperience-2