Geometry‑Grounded Video Diffusion for 3D‑Consistent World Generation

Recent advances in camera‑controlled video diffusion have enabled novel‑view synthesis from a single image and a user‑specified camera trajectory, but most methods rely on implicit spatio‑temporal representations and suffer from geometric implausibility, object distortion, and cross‑view inconsistencies when the viewpoint changes dramatically.

DreamWorld: Geometry‑Grounded Video Diffusion

DreamWorld (ECCV 2026) tackles these issues by explicitly grounding the generation process in 3D geometry. The core idea is to treat geometry as an intermediate structural pivot: a 3D foundation model provides multi‑scale geometric priors, which are compressed by a lightweight MLP‑based projection module and used as explicit conditioning for subsequent video synthesis.

Two‑Stage Generation Framework

Stage 1 – Geometry Video Diffusion : Given an initial image and a target camera trajectory, the system builds a point‑cloud representation of the observed scene, warps it to the new viewpoint, and extracts incomplete geometry features with the 3D foundation model. A geometry diffusion model then performs feature‑level flow matching to generate complete geometry representations for the target view.

Stage 2 – Appearance Video Diffusion : The completed geometry features are concatenated with noisy video latents (produced by a pretrained VAE encoder) along the channel dimension and fed to an appearance diffusion model that focuses solely on high‑fidelity RGB synthesis.

Key Design Choices

Geometry‑then‑Appearance decoupling separates structural completion from texture generation, allowing each stage to specialize.

The geometry stage emphasizes structural consistency, cross‑view alignment, and camera adherence.

The appearance stage concentrates on photorealistic texture and visual detail.

Quantitative Evaluation

DreamWorld was benchmarked on RealEstate10K, Tanks‑and‑Temples (Easy/Hard), and WorldScore. It achieved leading scores on novel‑view synthesis for both RealEstate10K and Tanks‑and‑Temples, and obtained an average WorldScore of 75.04, outperforming prior methods on 3D consistency and photo consistency metrics.

Ablation Study

Three variants were compared: (1) a baseline that conditions on warped partial images without explicit geometry; (2) a version without geometry diffusion that uses incomplete geometry features directly; (3) a version without geometry‑appearance decoupling that jointly models both. The full DreamWorld pipeline, which combines geometry diffusion with the two‑stage design, consistently achieved the best performance.

Conclusion

By rethinking camera‑controlled video generation from a 3D‑consistency perspective and introducing explicit geometry priors from a 3D foundation model, DreamWorld demonstrates that decoupling geometry and appearance yields superior visual quality, stronger cross‑view consistency, and better adherence to camera motion.