OpenWAM: Open-Source World-Action Model Stack for Robotics

Researchers from seven universities — including the National University of Singapore, Tsinghua University, and Peking University — have released OpenWAM, an open-source research stack for World-Action Models (WAM). The project provides a modular infrastructure (OpenWAM-Infra), a controlled experimental study (OpenWAM-Study), and a foundation model (OpenWAM-α) trained on large-scale human egocentric video and robot trajectory data.

OpenWAM-Infra: Modular Experimental Infrastructure

OpenWAM-Infra decouples the model, training runtime, deployment runtime, and evaluation protocol into four components with explicit interfaces. The model layer composes a visual encoder, interchangeable data-flow backbones, and visibility attention masks. Three architecture families are supported: single-system (shared backbone for video and action), dual-system (separate backbones connected via joint self-attention, cross-attention, or inverse dynamics), and triple-system (adding a vision-language understanding stream). All architectures are assembled via a unified configuration and registry mechanism, so the trainer and policy server remain agnostic to the current structure.

The unified runtime handles pre-training, fine-tuning, and checkpoint resumption, saving full configuration, module reconstruction information, and action normalization statistics. Deployment uses a single policy server supporting synchronous or asynchronous inference, multiple denoising schedules, and acceleration techniques. Evaluation connects lightweight clients to the server, enabling identical input-output interfaces across simulation and real robots. The current evaluation protocol covers eight simulation benchmarks — LIBERO, LIBERO-Plus, VLABench, RoboTwin2.0, RoboDojo, RoboCasa365, EBench, and RoboCasa-GR1 — spanning single-arm, dual-arm, mobile manipulation, and dexterous hand embodiments.

OpenWAM-Study: Controlled Experiments Yield Three Design Principles

Using OpenWAM-Infra on RoboTwin2.0 with fixed training budget and evaluation, OpenWAM-Study systematically investigates three research questions: what world knowledge to inherit, how to establish effective world-action synergy, and how to scale synergy across domains and embodiments.

Principle 1: Inherit a Strong Generative World Prior with Compact, Information-Dense Visual Latents

Four video generation backbones were compared: Wan2.1-VACE-1.3B (90.14% avg success), Cosmos-Predict2.5-2B (91.64%), Wan2.2-TI2V-5B (92.39%), and Wan2.1-I2V-14B (93.79%). Larger capacity consistently improved performance, but the 5B model was chosen as default for its cost-effectiveness (only 1.40% below the 14B model). The paper notes that the four models differ in architecture, data, and objectives, so gains cannot be attributed solely to parameter count.

Visual representation experiments show that the key is not whether an encoder is reconstruction-based or representation-based, but whether the latent space is compact and retains rich world information. DINOv3 and V-JEPA 2.1 produce high-dimensional features that initially underperform, but after compression via S-VAE to a DiT-compatible space, performance rises significantly; DINOv3 + S-VAE approaches Wan2.2-VAE. This yields the first principle: WAM needs a strong generative backbone and a visual latent space that is compact in both temporal and token dimensions with high information density.

Principle 2: Establish Explicit World-to-Action Information Flow During Training; Use Synchronous Joint Denoising at Inference

Architecture ablation shows that moving from single-system to dual-system and triple-system improves average success rates; dual-system joint self-attention reaches 92.36%, triple-system joint self-attention 92.60%. Balancing performance, complexity, and variable isolation, dual-system joint self-attention is selected for subsequent experiments.

Information flow masking experiments on RoboTwin2.0-Full reveal that isolated masking (87.41%) and "video sees action" (87.63%) perform poorly, while "action sees world" achieves 92.39% and bidirectional visibility 92.15%. Thus, action learning must have access to world information for effective synergy.

Denoising schedule comparison tests synchronous denoising against asynchronous variants where video or action leads. Synchronous denoising performs best; no asynchronous scheme consistently outperforms it. The second principle: training must establish explicit world-to-action information flow, and inference should maintain synchronous joint denoising along the joint noise plane.

Principle 3: Robot Data Provides Action Grounding; Human Egocentric Data Expands World Coverage; Single-Stage Joint Training Integrates Both Effectively

Cross-embodiment pre-training experiments under a 600-hour budget compare four recipes: from scratch, robot data only, human egocentric then robot data (staged), and human+robot joint single-stage. In-domain improvements are modest, but out-of-distribution (Clean2Random) gains are substantial: from scratch 14.50%, robot pre-training 23.80%, staged human→robot 26.50%, joint single-stage 26.62%.

Robot trajectories supply executable action grounding, boosting in-domain performance; human egocentric videos provide broader visual and interaction distributions, improving OOD generalization. Single-stage joint training matches staged training while avoiding extra stage transitions, becoming the default. Pre-training also shifts information flow preference: from scratch, unidirectional world-to-action slightly leads; after embodiment pre-training, bidirectional visibility becomes more stable across in-domain and OOD settings.

OpenWAM-α: Scaling Principles to a Foundation Model

OpenWAM-α combines the above principles into a publicly releasable foundation WAM. It uses Wan2.2-VAE as frozen visual encoder, Wan2.2-TI2V-5B as video backbone, and a 1B-parameter ActionDiT for the action stream. The two streams interact via joint self-attention with bidirectional visibility; language instructions and proprioceptive state condition the model through cross-attention.

To share a single action head across embodiments, OpenWAM-α defines an 80-dimensional unified action space: two mirrored 34-D arm blocks (each containing end-effector position, 6D rotation, gripper, and dexterous hand channels) plus 12 reserved slots for mobile base and other embodiment-specific channels. Each dataset maps its actions and states to fixed slots; unused dimensions are masked out during training.

Pre-training data sources: human egocentric video (~30%), synthetic robot data (~30%), and three real robot datasets (~40% combined). Raw data: ~1.33B frames (~14,300 hours); after visual quality filtering, robot signal cleaning, and source-balanced sampling, the training set comprises ~518.5M frames (~6,369 hours). Human data includes 71.6K long egocentric recordings covering 3,006 daily manipulation tasks; real robot data covers 17 physical platforms.

Deployment uses synchronous inference with 10-step joint denoising. Optimizations — fixed-shape compilation, CUDA Graph, speed cache, prompt embedding cache, and skipping VAE decoding in the control path — yield ~170 ms per action chunk on an RTX 5090.

Simulation Benchmark Results

OpenWAM-α is evaluated on eight benchmarks covering five embodiment types. It ranks in the top tier on LIBERO, VLABench, RoboTwin2.0-Full, RoboCasa365, and RoboCasa-GR1. On the mobile dual-arm benchmark EBench, it achieves the current best Score and Success Rate, leading the second place by ~4 percentage points. On RoboTwin2.0-Clean2Random and RoboDojo, it trails the best VLA methods but is the strongest among WAM approaches.

LIBERO-Plus is an exception: performance drops mainly on camera and noise perturbations, also affecting background and layout perturbations. The paper attributes this to limited pre-training data matching single-arm perturbation tests and the sensitivity of long-horizon future prediction in pixel latent space to viewpoint and noise changes. This highlights that generalization under specific test conditions depends on whether pre-training covers similar embodiments and environmental variations.

The paper compares VLA vs. WAM: no universal winner exists. WAM benefits from future video latent supervision, fitting in-distribution tasks more closely; VLA avoids long-horizon future prediction and is often more robust to OOD perturbations. Both limitations are data-related; richer, more diverse pre-training data with precise action annotations can improve both fitting and generalization.

Real Robot Validation

Real-robot experiments on three embodiments: Franka-Research-3 single arm, RoboDojo dual-arm (ARX X5, Piper, Piper X), and Wuji dexterous hand + Tianji arm. Single-arm tasks: stacking, pick-place, hanging (6 tasks, avg success 82.5% vs. LingBot-VA 77.5% and π0.5 55.0%; two tasks reach 100%). RoboDojo dual-arm: overall average score 37.6, avg success 24.4%, leading on most tasks across three platforms.

Dexterous hand experiments test unseen embodiments: Wuji hand (21 DoF) + Tianji arm (9-DoF end-effector pose) — neither appeared in pre-training. After fine-tuning, OpenWAM-α clearly outperforms π0.5 on toy tower stacking, badminton collection, clothing storage, and bottle cap twisting in both in-domain and OOD settings, demonstrating that the unified action space and pre-training prior enable adaptation to new robot morphologies.

Significance of the Open Research Stack

OpenWAM contributes more than model scores. OpenWAM-Infra unifies model, training, deployment, and evaluation under common interfaces; OpenWAM-Study converts empirical judgments into verifiable design principles via controlled experiments; OpenWAM-α scales those principles to multi-source, multi-domain, multi-embodiment data. Together they form a complete pipeline from research questions to reusable models.

The release includes infrastructure code, evaluation protocols, pre-trained weights, and data recipes, aiming to provide a reproducible common starting point for WAM research. The authors acknowledge that future work requires larger, more diverse, and precisely annotated embodiment data; more compact and environment-robust visual representations; and end-to-end designs that fuse WAM and VLA strengths.

Links: ArXiv: https://arxiv.org/abs/2609.07398; Project page: https://openwam-official.github.io/; Code: https://github.com/OpenWAM-Official/OpenWAM; Models & data: https://huggingface.co/OpenWAM