How VChain Gives Video Generation a Visual Thought Chain for Explicit Spatiotemporal Planning

When video generation models push visual fidelity, a key bottleneck emerges: do they truly understand the real world and can they reason about plausible evolution? This paper addresses that gap by proposing VChain, a framework that leverages the visual reasoning abilities of large multimodal models (e.g., GPT‑4o) as an "external brain" for video generation.

Background

Current video generators often exhibit "physical failures"—balls roll unnaturally, feathers fall faster than stones—because they excel at mimicking appearance but lack physical causality. Directly using multimodal large models for video synthesis is prohibitively expensive.

Method

VChain operates entirely at inference time in three stages, requiring no retraining of the video generator.

Visual Thought Reasoning : Given a textual instruction (e.g., "pour concentrated sulfuric acid onto a wooden table"), the LMM brainstorms a causal chain and produces a series of key frames called the Chain of Visual Thoughts . The model iteratively imagines each sub‑step—acid hovering, pouring, contacting the surface, and corrosion—generating an image for each.

Sparse Inference‑Time Visual‑State Adaptation : The key frames and their captions form a sparse supervision signal. Using LoRA, VChain fine‑tunes the pretrained video generator only at these critical moments, dramatically reducing computation.

Video Sampling : After adaptation, the generator receives a long prompt that concatenates all step descriptions, producing a coherent, physically plausible video that follows the outlined outline.

Experimental Results

Qualitative comparison : In a "bowling ball hitting pins" scenario, baseline models either leave pins unmoved or produce jittery, artifact‑filled interactions. With VChain, the ball strikes the pins with realistic force, and the geometry and material properties remain consistent throughout.

Quantitative evaluation : VChain outperforms existing methods on dedicated benchmarks for physical laws, common‑sense reasoning, and causal logic, achieving higher scores across all metrics.

Ablation Study

Removing the visual‑thought component causes the model to miss correct visual patterns (e.g., proper catching perspective). Omitting sparse adaptation leads to severe image distortion and artifacts when interpolating frames. Only the combination yields the most coherent and realistic results.

Discussion

VChain is an "plug‑and‑play" inference‑time framework that does not require training a new video model nor additional data; it simply empowers the generator with LMM reasoning. This illustrates a new "Reasoner‑Renderer" collaboration paradigm, where complex logical judgment (handled by the multimodal LLM) is decoupled from low‑level visual rendering (handled by diffusion‑based or transformer video models).

By moving from purely semantic guidance to concrete visual reasoning, VChain demonstrates that video generation benefits from "de‑symbolizing" the reasoning process and anchoring it in visual space, paving the way for future multimodal model cooperation.