Why Robots Need World Models: A Joint Survey from Leading Institutions

World models are becoming indispensable in robot learning as the field moves from task‑specific policies to more general Vision‑Language‑Action (VLA) models that unify visual observations, language instructions, and action outputs.

In real‑world settings, robots must handle contact, occlusion, long‑term dependencies, error accumulation, and multi‑step planning, which simple perception‑to‑action mappings cannot address. A world model predicts how the environment will evolve after a candidate action, providing the foresight needed for robust control.

The authors, representing Nanyang Technological University MARS Lab, UC Berkeley, Stanford, Harvard, Princeton, ETH Zurich, Oxford, Tokyo University, and Microsoft, released the 43‑page survey World Model for Robot Learning: A Comprehensive Survey (arXiv:2605.00080) together with a continuously updated GitHub repository (

https://github.com/NTUMARS/Awesome-World-Model-for-Robotics-Policy

) that systematically reviews definitions, architectural paradigms, application scenarios, evaluation benchmarks, and future challenges.

In robotics, a world model is defined as a predictive model that describes the agent‑environment dynamics: given the current state and a proposed action, it predicts the next state. This distinguishes it from generic video‑generation models, which may produce visually plausible frames without preserving action consistency or physical realism.

The survey groups the core capabilities of robot world models into three categories: (1) foresight – predicting action consequences before execution; (2) imagination‑driven planning – comparing candidate behaviors via imagined rollouts; and (3) data amplification – synthesizing trajectories or demonstrations to improve policy learning.

Integration with VLA strategies has evolved through several architectural families:

Decoupled two‑stage pipelines (e.g., UniPi, VidMan, Vidar, Gen2Act) first predict future observations and then use an inverse‑dynamics module to infer actions, offering clear modularity but suffering from interface errors.

Single‑backbone approaches (e.g., UVA, UWA, VideoVLA, Cosmos Policy) embed visual prediction and action generation within a unified diffusion or flow‑matching network, reducing latency and error propagation.

Mixture‑of‑Experts / Mixture‑of‑Tasks designs (e.g., Motus, LingBot‑VA, BagelVLA) keep modality‑specific experts while sharing attention across branches, preserving specialized capabilities while enabling cross‑modal information flow.

Unified VLA methods (e.g., GR‑1, WorldVLA, DreamVLA, UniVLA, CoWVLA) internalize future‑state prediction directly into the VLA training objective, eliminating the need for an external world‑model module.

The authors note that no single route dominates; performance depends on data scale, control frequency, task complexity, inference cost, and the model’s ability to capture action‑conditioned physical changes.

Another major direction treats the world model as a simulator ( World Model as Simulator ). Here the model receives the current observation, task instruction, and candidate actions, then predicts the next observation, reward, or termination signal, allowing reinforcement‑learning agents to train in a learned environment. However, errors in dynamics prediction can be amplified over multi‑step rollouts, making stability, action sensitivity, and reward consistency critical concerns.

Large‑scale video‑generation models have recently been explored as foundations for robot world models. The survey outlines a progression from imagination‑based generation, to action‑controllable models, to structure‑aware models that incorporate depth, 3D geometry, and object representations, and finally to foundation‑scale models with massive data and multi‑task generalization.

Evaluation metrics are shifting from open‑loop visual fidelity toward closed‑loop task utility. Benchmarks should measure whether a world model improves real‑task success rates, correctly ranks candidate actions, predicts failure trajectories, maintains causal consistency over long horizons, and reduces the number of real‑world interaction samples required.

The paper catalogs several robot‑learning benchmarks and datasets—including LIBERO, RoboTwin, CALVIN, and SIMPLER—and reports that the most effective methods vary across tasks, with decoupled, unified, expert‑mix, and latent‑space approaches all showing competitive results.

Future challenges highlighted include ensuring causal consistency under actions, achieving real‑time inference efficiency (especially for video diffusion models), and incorporating physical grounding such as force, tactile feedback, and structured geometry. The authors also stress that neural world models will likely complement rather than replace symbolic planning and classical control, suggesting hybrid systems as a promising research direction.

In conclusion, the survey argues that world models should serve concrete control purposes—assisting policy generation, acting as simulators for training, supporting evaluation and planning, and generating synthetic data—rather than merely imagining visually plausible futures.