The First Industry Survey of Vision World Models: Toward a Higher‑Intelligence Visual Paradigm

Why a Vision World Model Survey Is Needed

World models have become a focal point of AI research, influencing video generation, representation learning, embodied intelligence, and autonomous driving. Although many approaches rely on visual inputs, they often treat vision merely as an observation modality, leading to fragmented definitions, inconsistent taxonomies, and misaligned evaluation standards.

Unified Vision World Model (VWM) Framework

The authors define a Vision World Model (VWM) as a system that learns world knowledge from visual data and generates future world states conditioned on interaction inputs. They organise VWM research into three core components:

Vision Encoding : Transform raw visual signals (images, video, point clouds, optical flow) into representations suitable for modelling world dynamics.

Knowledge Learning : Capture three progressive layers of knowledge – spatio‑temporal coherence, physical dynamics, and causal mechanisms.

Controllable Simulation : Use the learned knowledge to simulate future states under interactive conditions such as robot actions or textual commands.

This framework answers the questions of what a VWM should learn, how it should learn, and how it can be controlled and evaluated.

Technical Routes of Vision World Models

Based on the unified framework, existing methods are grouped into four representative paradigms, each with sub‑paradigms:

Sequential Generation : Tokenise images/video and predict future tokens autoregressively. Advantages: scalability and long‑context handling. Limitations: error accumulation, limited fine‑grained physical simulation.

Diffusion‑Based Generation : Iteratively denoise in latent space to generate future frames. Produces higher visual fidelity but incurs higher inference cost.

Embedding Prediction : Directly predict future embeddings rather than full frames, focusing on learning dynamics for planning and reasoning; however, interpretability is weaker.

State Transition : Compress visual input into compact latent states and model temporal evolution via recurrent transitions, enabling efficient rollout and long‑term memory.

Evaluation Metrics and Benchmarks

A reliable VWM must satisfy three criteria: visual realism, physical plausibility, and task performance. Accordingly, the authors organise metrics into three categories:

Visual Quality – clarity, smoothness, and realism of generated media.

Physical Plausibility – adherence to physical laws, consistent 3D structure, and multi‑view coherence.

Task Performance – effectiveness in downstream tasks such as robotic grasping or autonomous‑driving safety.

Benchmarks are divided into two groups:

Foundational World Modeling – general physics and causality benchmarks that test universal world understanding.

Domain‑Specific World Modeling – datasets for embodied AI, autonomous driving, and interactive gaming.

Future Directions: The 3R Roadmap

The authors identify three crucial breakthroughs for next‑generation VWMs:

Re‑grounding : Expand knowledge beyond simple physics to include complex material dynamics, social norms, and intent‑driven causality; upgrade architectures with geometry‑aware and neuro‑symbolic components.

Re‑evaluation : Deploy judge models and real‑world robotic trials to expose physical failures; employ counterfactual reasoning tests to assess causal understanding.

Re‑scaling : Scale pre‑training with massive, causally rich interaction data and develop inference‑time mechanisms that allow internal deliberation, multi‑hypothesis generation, and self‑correction before output.

Conclusion

Vision world models aim to break the limits of symbolic language models by directly modelling the continuous physical and causal evolution of the world, representing a paradigm shift toward AI systems capable of predictive, interactive, and decision‑making intelligence.