How VGGRPO Uses 4D Latent Rewards for World‑Consistent Video Generation

Problem

Large‑scale video diffusion models achieve high visual quality but often exhibit geometric drift, unstable camera trajectories, and incoherent scene structures. These issues hinder downstream tasks such as embodied intelligence and world‑action modeling that require stable camera motion and consistent geometry.

Prior Approaches

Two main strategies have been explored:

Modifying the generator architecture with geometry‑aware modules, which can degrade the generalization of internet‑scale pretrained models.

Applying reinforcement‑learning‑based post‑training alignment that relies on RGB‑space rewards. This requires repeated VAE decoding, incurs high computational cost, and fails to generalize to highly dynamic real‑world scenes.

VGGRPO Overview

VGGRPO (Visual Geometry GRPO) introduces a geometry‑aware post‑training framework that operates entirely in latent space, preserving the pretrained model’s generalization while improving geometric consistency for dynamic scenes.

Latent Geometry Model (LGM)

An LGM connects the diffusion model’s latent variables to a pretrained geometry foundation model via a lightweight adapter. The adapter enables direct decoding of 4‑D scene geometry (point clouds, depth, camera pose, optical flow) from latent space, eliminating the need to reconstruct RGB pixels before geometry extraction. Because the geometry foundation model supports 4‑D reconstruction, LGM naturally extends to dynamic scenes.

Reward Design and GRPO Optimization

VGGRPO applies Group‑Relative Policy Optimization (GRPO) in latent space with two complementary reward components:

Camera Motion Smoothness Reward : Uses the camera pose predicted by LGM to compute translational and rotational accelerations. High acceleration is penalized, encouraging smooth, physically plausible camera trajectories.

Geometry Reprojection Consistency Reward : Reprojects the predicted 3D structure into multiple views using point clouds, depth maps, camera parameters, and optical flow. Depth differences across views are measured, and dynamic regions are filtered out via optical flow, ensuring cross‑view geometric coherence.

Both rewards are computed without any VAE decoding, removing the expensive pixel‑level reward step of prior methods.

Experiments

Qualitative comparison (Figure 1) shows that VGGRPO markedly reduces geometric drift and structural artifacts in challenging dynamic scenes compared with baseline models.

Quantitatively, VGGRPO outperforms baselines on static and dynamic benchmark suites across geometry‑related metrics (e.g., depth consistency, pose smoothness) and overall video quality scores, demonstrating that geometry‑aware post‑training does not sacrifice generative fidelity.

Ablation Study

When only the motion‑smoothness reward is applied, camera trajectories become stable but geometric artifacts remain (highlighted in Figure 4). Adding the geometry‑reprojection reward further improves scene consistency while preserving smooth motion, confirming the complementary nature of the two rewards.

Significance

VGGRPO achieves world‑consistent video generation without architectural modifications or costly pixel‑level rewards. By injecting geometric priors directly into latent space and optimizing with 4‑D rewards, the method produces videos that are both visually realistic and physically consistent, benefiting applications such as world‑action modeling and embodied AI.

Paper title: VGGRPO: Towards World‑Consistent Video Generation with 4D Latent Reward

Paper page: https://zhaochongan.github.io/projects/VGGRPO/

ArXiv PDF: https://arxiv.org/pdf/2603.26599