DeepMind’s GenCeption Shows Video Generation Can Serve as a General‑Purpose Vision Learner

Traditional video‑understanding pipelines decompose a task into multiple stages—pixel extraction, pose estimation, temporal segmentation—and rely on task‑specific annotations, which leads to misaligned data, error accumulation, and the need to redesign the system for each new task.

GenCeption: Turning Generation into Understanding

In the paper Video Generation Models are General‑Purpose Vision Learners , DeepMind introduces GenCeption , a method that directly reuses a pretrained text‑to‑video diffusion model. Instead of feeding random noise, the model receives a noise‑free latent representation of the input video and performs a single forward pass (t=0), converting the diffusion process into a deterministic feed‑forward network.

By conditioning on a textual instruction, the same model can output different modalities: depth maps, 3D keypoints, surface normals, segmentation masks, or camera trajectories. Dense tasks are encoded as RGB videos, while sparse tasks (2D/3D keypoints) use learnable tokens decoded by a lightweight MLP. All tasks share the same backbone, prediction head and a unified L2 loss; differences are expressed only through data format.

Unified Data Representation

The authors argue that a universal pre‑training objective—next‑token prediction—must satisfy three conditions: (1) it learns temporal evolution, 3‑D structure and physical laws; (2) it aligns naturally with language; (3) it scales with data, parameters and compute. Large‑scale text‑to‑video generation meets these criteria.

To solve the annotation‑alignment problem, GenCeption relies on synthetic data. Using 800 RenderPeople assets and 200 actions, the team generated 7,500 human videos together with synchronized depth, normal, segmentation, DensePose, 2D/3D keypoints and camera pose labels. Because all signals originate from the same 3‑D scene, they are perfectly aligned in space and time.

Experimental Results

GenCeption was evaluated in two versions: a specialist model trained on a single task and a generalist model jointly trained on all tasks. The performance gap between them was small, indicating that multi‑task training did not cause significant interference. On benchmarks for depth, segmentation, pose and surface normal, the generalist matched or outperformed dedicated models such as Depth Anything V3, SAM‑3, D4RT, VGGT‑Ω, Sapiens and Genmo.

Data efficiency was striking: GenCeption achieved comparable results with only ~1.23 M training frames, whereas specialist models required 60 M–600 M frames. Scaling the backbone from 1.3 B to 14 B parameters further improved accuracy, especially on depth estimation where the WAN‑2.1 pretrained backbone outperformed VideoMAE‑V2 and V‑JEPA.

Beyond the training distribution, the model generalized to multi‑person scenes, unseen animal categories and robotic embodiments without additional supervision, suggesting that the large‑scale video generation pre‑training had already captured extensive world knowledge about space, time, objects and motion.

Discussion: Does “generation = understanding” Hold?

The study provides strong evidence that video‑generation pre‑training yields representations useful for a wide range of visual‑understanding tasks. However, the authors caution that structural knowledge (geometry, motion) is well captured, while higher‑level semantics such as action intent or causal reasoning remain open challenges.

Overall, the work proposes a new development paradigm: given a powerful video‑generation foundation model, downstream tasks can be tackled with minimal task‑specific data by simply defining an appropriate output format and textual prompt.