World Labs Unveils Three 3D Generation Papers While Co‑Founder Announces Departure

Background: 3D Generation Challenges

Training data for 3D tasks are scarce because most available data are 2D images or videos. Geometry consistency becomes exponentially harder when moving from 2D to 3D. Recent work therefore transfers the strong priors of 2D diffusion models to 3D generation instead of training 3D generators from scratch.

World Tracing

Paper title: World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible

Each input pixel is treated as a ray and an ordered stack of 3D points is predicted along the ray. Layer 0 corresponds to the visible surface; deeper layers encode hidden geometry. This “pixel‑aligned multilayer geometry representation” is modeled with a diffusion process that remains aligned to the original pixel coordinates, anchoring visible depth precisely while generatively completing occluded regions. The method therefore reconstructs both visible and hidden geometry from a single image.

Project page: https://haoz19.github.io/world-tracing-page/

Modality Forcing

Paper title: Modality Forcing for Scalable Spatial Generation

A text‑to‑image diffusion model is fine‑tuned to handle both RGB and depth modalities by assigning independent per‑modality noise levels during training. Each modality receives its own noise schedule and loss. At inference, fixing the noise level of one modality to zero yields conditional generation of the other (image‑to‑depth or depth‑to‑image); joint noise produces simultaneous RGB‑D synthesis. This unified approach removes the need for a separate depth network or task‑specific architectural branches.

Project page: https://modality-forcing.github.io/

Flex4DHuman

Paper title: Flex4DHuman: Flexible Multi‑view Video Diffusion for 4D Human Reconstruction

The method builds on Alibaba’s Wan 2.1, a 1.3 B‑parameter video DiT. The original spatio‑temporal position encoding is replaced with a five‑axis encoding that adds a view‑slot index and continuous SE(3) relative camera geometry. This enables the model to generate synchronized multi‑view videos from a single monocular clip without requiring skeletons, depth maps, or normal priors, unlike methods such as Diffuman4D that depend on SMPL skeletons.

Generated multi‑view videos are fed to a FreeTimeGS pipeline to produce dynamic 4D Gaussian splats.

Benchmarks:

DNA‑Rendering: PSNR improvement of ~9.3 dB over Diffuman4D‑mono‑skeleton.

ActorsHQ (zero‑shot): PSNR improvement of ~3.4 dB over the same baseline.

After minimal fine‑tuning, the model also generalizes to multiple animal species, demonstrating that the design does not rely on human‑specific geometry.

Project page: https://andy-cheng.github.io/Flex4DHuman/