Is Your World Model Too Slow? Fast‑LeWM Boosts Dynamic Prediction by 4× with Action‑Prefix Parallelism

In visual planning and embodied AI, world models are regarded as the core component that lets an agent imagine future latent states before executing actions. However, the imagination step is often a bottleneck because each candidate action sequence is evaluated by a step‑by‑step autoregressive rollout, leading to slow planning and error accumulation along the imagined trajectory.

Fast‑LeWM (Fast LeWorldModel) addresses these issues by fundamentally changing the prediction paradigm: instead of rolling out one latent transition at a time, it predicts an entire trajectory in parallel using action‑prefix parallel prediction . The model consists of three parts:

Visual Encoder : maps the current observation and future observations into a latent space, producing the current latent z_t and the target future latent as supervision.

Action‑Prefix Encoder : encodes each candidate action sequence with a causal Transformer into a set of prefix tokens, where the k‑th token contains information of the first k actions. A state token derived from z_t is prepended to provide context.

Parallel Latent Predictor : consumes the current latent and all prefix tokens to output all future latents in a single forward pass.

During training, dense supervision is applied to the prediction of every prefix, not only the final state, and a SIGReg collapse‑prevention regularizer is retained. An optional self‑consistency term weighted by β penalizes discrepancies between the direct prediction of a long‑horizon latent and the two‑step prediction via an intermediate latent.

Experiments follow the same goal‑conditioned latent planning protocol as the original LeWorldModel on Four environments (Two‑Room, Reacher, PushT, OGBench‑Cube). Results show that Fast‑LeWM raises average success rate from 85.8% to 90.5% and to 92.0% when the self‑consistency term is enabled. Dynamics computation time drops from 31.4 s to 8.0 s (≈4× faster), and total CEM solve time falls from 54.4 s to 28.3 s, a 48% reduction.

Additional ablation studies reveal that simply lengthening the action block (Long‑Action LeWM) does not match the performance of the prefix‑based approach, and removing the state token degrades results, confirming the importance of both the prefix representation and dense supervision.

The authors conclude that the primary bottleneck for planning‑oriented world models lies in the interface design rather than the model itself; parallel generation of future states via action‑prefix prediction offers a more efficient path toward high‑performance visual planning.