VBVR-Pro: 300 Visual Reasoning Tasks, Verifiable Rewards, and a Unified Training Framework

Researchers from NTU, CMU, and Berkeley have released VBVR-Pro, a scalable and verifiable suite for native visual reasoning. The system addresses three core gaps: what tasks to train on, how to evaluate generated images or videos reliably, and which visual modality (image, video, or interleaved image-text) best supports reasoning.

300 Tasks, 1.25M Samples, Dual Modality Data

VBVR-Pro defines 300 visual reasoning tasks spanning five core capabilities: perception, spatial reasoning, transformation, abstraction, and knowledge. These tasks are rendered into 1.25 million high-quality training samples, each provided in both video and interleaved image-text formats. This dual-modality dataset enables fair comparison across different model architectures under identical configurations.

Despite focusing on abstract visual scenes, models trained on VBVR-Pro show strong cross-domain generalization. Evaluated on seven external benchmarks — RISE-Video, V-ReasonBench, RULER-Bench, MME-CoF-Pro, BabyVision, and others — the trained models achieve up to 20 percentage points improvement on unseen real-world and embodied reasoning tasks, indicating learning beyond visual memorization.

Verifiable Scorers Replace VLM-as-a-Judge

Current benchmarks often rely on VLM-as-a-Judge, which suffers from imprecise visual attribute assessment, overlooking critical local errors, misunderstanding task rules, high cost, and instability. VBVR-Pro-Bench implements dedicated verifiable scorers for 100 tasks. Each scorer extracts task-relevant semantic information — object color, shape, position, trajectory — and applies carefully designed scoring rules. For example, in a maze task the scorer explicitly checks path correctness and wall-crossing violations, yielding interpretable scores.

Large-scale human preference experiments on 4,000 videos from eight models show scorer-human agreement exceeds 60%, outperforming GPT-5.5 (0.54) and Gemini-3.1-Pro (0.52), and approaching the human inter-annotator ceiling of ~0.77.

Modality Comparison: Image, Video, or Interleaved?

Over 30 open and closed-source generative models were tested; nine open-source models were further trained on VBVR-Pro data. All models improved significantly. The study finds: single images suit tasks focused on final state without complex intermediate processes; interleaved image-text is efficient and competitive when discrete states suffice to express reasoning; video shows clear advantage when tasks depend on continuous state changes. This suggests different visual carriers fit different reasoning problems.

Interestingly, models do not rely heavily on explicit textual chain-of-thought. Removing or perturbing intermediate text preserves reasoning performance, whereas removing or corrupting intermediate visual states causes significant drops. This indicates intermediate visual states are not merely output displays but essential carriers for organizing and advancing the reasoning process.

Both video and interleaved models exhibit a Chain-of-Step (CoS) phenomenon: during generation they explore multiple candidate visual states in intermediate denoising steps before converging to the final answer. In path planning, for instance, the model first explores multiple routes then gradually settles on the feasible shortest path.

From Judge to Coach: Verifiable Rewards for Reinforcement Learning

The verifiable scorers close the loop by serving as low-cost, accurate rewards for large-scale reinforcement learning. Using VBVR-Pro-Wan2.2-TI2V-5B as baseline (score 0.470 on VBVR-Pro-Bench), supervised fine-tuning on 50k samples raises the score to 0.503, and subsequent RLVR with verifiable rewards further improves it to 0.548 — the best result under the same data budget. RLVR also boosts out-of-domain task performance.

Beyond final scores, RLVR changes the reasoning dynamics. In a color sequence completion task, the pre-RL model locks onto an incorrect answer early and maintains it; the RLVR-trained model continues to adjust visual states in later denoising steps, eventually converging to the correct answer. This demonstrates self-correction capability induced by verifiable rewards.

Conclusion

VBVR-Pro provides a unified experimental foundation for visual reasoning: 300 tasks, unified multimodal data, verifiable benchmark, systematic comparison of 30+ models, and verifiable rewards usable for large-scale RL. The work suggests that a model's "chain of thought" need not reside only in text — some problems may truly require visual thinking.