ABot-M0.5: The First Unified World Action Model for Mobile Manipulation

Problem: Structural mismatches in existing World‑Action Models (WAMs) for mobile manipulation

Mobile manipulation requires both navigation and manipulation over long horizons. Current visual‑language‑action (VLA) approaches lack explicit world models, and emerging WAMs suffer from three structural mismatches:

Temporal granularity mismatch : coarse‑grained video prediction cannot represent micro‑dynamics such as fingertip contact.

Action‑space coupling : low‑frequency base motion and high‑frequency arm control follow different dynamics, yet are forced into a single action space.

Train‑test distribution mismatch : during deployment the model conditions on its own predicted future observations, causing exposure bias and error accumulation.

ABot‑M0.5: Unified World‑Action Model for mobile manipulation

ABot‑M0.5 aligns video prediction, intermediate latent actions, and low‑level physical control. It introduces three core technical components.

1. Intermediate latent actions

Mechanism : a frame‑level latent‑action space is inserted between video latent variables and physical actions. Self‑supervised algebraic constraints (additive consistency for long‑term transitions and forward‑backward reversal consistency) enforce a continuous, hardware‑agnostic motion representation.

Benefit : the latent layer translates visual dynamics into high‑frequency control signals, preserving fine‑grained details such as contact events.

2. Dual‑level Mixture‑of‑Transformers (MoT)

Mechanism : separate Transformer sub‑towers handle base mobility and arm manipulation. Each modality (video, latent actions, executable actions) receives its own projection head, and the two action streams are processed in distinct sub‑towers.

Benefit : decouples low‑frequency base motion from high‑frequency arm control, eliminating interference while sharing low‑level visual features for coordinated objectives.

3. Dream Forcing training

Mechanism : in later training stages ground‑truth video latents are cut off; the model must condition on its own predicted video (“dream”) to generate actions.

Benefit : forces the model to learn robust control under imperfect predictions, aligning training and inference distributions and dramatically reducing error accumulation.

Three‑stage progressive training

Stage 1 – World model pre‑training : initialize from the Wan2.2 video model, fine‑tune on large multi‑embodiment datasets (OXE, Agibot‑Beta, Galaxea, RoboCOIN) using a conditional flow‑matching loss to learn physical dynamics in latent space.

Stage 2 – Latent‑action pre‑training : train a frame‑level latent‑action encoder on massive unlabeled video corpora with the algebraic constraints, then freeze it as a universal motion feature extractor.

Stage 3 – Progressive supervised fine‑tuning :

Sub‑stage 1 (teacher‑forcing “greenhouse”): use ground‑truth future video latents to jointly optimise video, latent, and physical action predictions.

Sub‑stage 2 (Dream Forcing): cut off ground‑truth and condition on self‑generated video, eliminating train‑test distribution shift.

Comprehensive evaluation

Long‑horizon success (RoboCasa365) : ABot‑M0.5 achieves 40.4% average success, surpassing Diffusion Policy (6.1%), π0 (14.8%) and Qwen‑RobotManip (35.9%). With the compression‑memory variant ABot‑M0.6 the success rises to 46.6%.

Fine‑grained precision :

RoboTwin 2.0 – 94.10% average success (Clean Easy 94.0%, Randomized Hard 94.2%).

LIBERO – average success across L‑Spatial, L‑Object, L‑Goal, L‑Long reaches 99.4% (new SOTA).

LIBERO‑Plus zero‑shot under camera, robot, light, and noise disturbances – total average success 83.4%, outperforming prior WAMs.

Real‑world deployment (Agilex Piper) :

Peg‑cylinder task – 70% success and 96% process score (π0: 50%/90%; Fast‑WAM: 30%/77%).

Multi‑stage tasks (Organize Plate, Arrange Fruits, Cup Stacking, Arrange Flower) – success rates 70%, 80%, 80%, 60% respectively, all with process scores > 88%, far exceeding Fast‑WAM’s 20‑40% success.

Paper: https://arxiv.org/pdf/2607.00678

Code: https://github.com/amap-cvlab/ABot-Manipulation