DeepMind’s Video Generation Model Becomes a General Visual Intelligence – He Kaiming’s Involvement

Visual AI’s Next Token

Just as large language models unified NLP by predicting the next token, computer vision still relies on task‑specific models. GenCeption proposes a universal visual pre‑training objective analogous to next‑token prediction, aiming to learn temporal evolution, 3D structure, and physical laws while staying aligned with language and scalable with data and compute.

One Prompt, One Architecture

GenCeption’s core idea is to reuse a pretrained text‑to‑video diffusion model (a 140 B‑parameter video generation model) as a feed‑forward visual backbone. Instead of starting from random noise, the model receives a noise‑free latent representation of the input video, fixes the diffusion timestep at t=0, and performs a single forward pass.

Because the model is conditioned on a textual instruction, changing the prompt switches tasks without altering the network weights. Unlike many multitask vision models that still require separate task heads and loss functions, GenCeption seeks to unify the backbone, prediction head, and training objective.

Unified Task Representation

Tasks are divided into two categories. Dense tasks (depth, surface normals, segmentation, camera rays) are encoded as RGB videos, allowing the model to output them using the same generation format it already knows. Sparse tasks (2D/3D keypoints) add learnable tokens that a lightweight MLP decodes into coordinates. All tasks share a single L2 loss; differences are expressed through data format rather than architectural changes.

For example, GenCeption can handle fast motion, severe occlusion, first‑person and multi‑view scenarios, outputting 4‑D human keypoints suitable for robotics or AR/VR.

Synthetic Data, Multitask Alignment

Real video datasets rarely contain all required annotations. To obtain high‑quality, perfectly aligned labels for many tasks, the authors generated synthetic data: 800 RenderPeople assets, 200 actions, producing 7 500 human videos with synchronized depth, normals, segmentation, DensePose, 2D/3D keypoints, and camera poses. This yields a unified 3‑D scene where spatial‑temporal alignment is inherent.

GenCeption achieves comparable or superior performance to specialized models such as Depth Anything V3, SAM 3, D4RT, VGGT‑Ω, Sapiens, and Genmo on multiple benchmarks.

Data efficiency is striking: after training on roughly 1.23 million frames, GenCeption reaches performance close to those specialized models that used 200 million–600 million frames (Depth Anything V3 ≈ 2 B, D4RT ≈ 86 M, VGGT‑Ω ≈ 600 M). This translates to a 7‑to‑500× reduction in downstream training data.

Generalisation Beyond Training

Although most training data are synthetic human videos, GenCeption successfully processes real videos, delivering depth, normals, segmentation, and pose outputs. It also generalises to multi‑person scenes despite being trained mainly on single‑person videos.

The model can transfer to unseen categories: a model fine‑tuned on synthetic humans can predict approximate human‑like poses for apes, segment animals, or provide robot‑relevant keypoints, demonstrating that the world knowledge acquired during large‑scale video generation pre‑training transfers to downstream visual tasks.

In summary, by turning a powerful text‑to‑video diffusion model into a unified feed‑forward visual backbone, GenCeption demonstrates that video generation can serve as a universal pre‑training task for vision, offering high performance across diverse tasks with minimal task‑specific data.

Reference: GenCeption (arXiv 2607.09024) – https://genception.github.io/