Can Generative Models ‘Draw’ Spatial Intelligence? Introducing the Agentic ProVisE Evaluation Framework

ProVisE framework

ProVisE (Protocolized Visual Evaluation) enables image‑generation models to answer spatial questions by rendering a visual answer and converting it back to the structured format required by existing benchmarks.

A visual protocol consists of a guidance prompt that specifies how the model should depict the answer (e.g., marking a target, filling a direction grid, producing a depth map, shading a region, drawing a robot trajectory) and a corresponding parser that extracts colors, positions, or regions from the generated image.

The inference pipeline has three steps:

Protocol Router selects an appropriate visual protocol for the given task.

The image‑generation model follows the guidance prompt and produces a parsable visual answer.

Protocol Parser translates the pixel‑level result into coordinates, masks, states, or trajectories required by the original benchmark.

Agentic Builder

The Agentic Builder automatically constructs visual protocols from task input, answer structure, and scoring rules. It operates in three modes:

Reuse : directly reuse an existing compatible protocol.

Build : assemble a new protocol from registered parsing components.

Fallback : when deterministic components cannot cover the task, an auxiliary VLM parses the generated image to recover a structured prediction.

After construction, each protocol is validated for generation, parsing, and metric integration; validated protocols are frozen for evaluating all image‑generation models.

Illustrative examples

Examples include highlighting a target bus, drawing a robotic arm’s trajectory on the original scene, generating a depth map to compare distances between points A and B, and adding a second passenger to a cable‑car seat to visualize spatial feasibility. Corresponding parsers convert these visual answers back into coordinates, option selections, or trajectories for scoring.

SpatialGen‑Bench

SpatialGen‑Bench comprises 14 sub‑tasks organized into four hierarchical levels:

Perception : counting, relative depth, orientation, size comparison.

Understanding : grounding, relation analysis, view judgment, scene modeling.

Reasoning : multi‑step inference, state prediction, geometric feasibility.

Interaction : affordance grounding, navigation, trajectory planning.

Benchmark repository: https://huggingface.co/datasets/wx91726/SpatialGen-Bench

Evaluation results

Twenty text‑output VLMs and eleven image‑generation models were evaluated on SpatialGen‑Bench.

Image‑generation models achieve the highest scores on tasks that can be answered directly with depth maps or spatial markings, reducing information loss from converting continuous space to text. The top generative model (GPT‑5 Image 2) scores 54.49 points.

Text‑output VLMs outperform generative models on abstract reasoning tasks (counting, size comparison, geometric feasibility, state prediction) with an average advantage of 17.6 percentage points on the reasoning tier.

Failure analysis shows that 88.03 % of erroneous generative samples still produced a parsable image; the error resides in the depicted spatial state rather than in image generation or parsing.

Conclusion

The results suggest that spatial intelligence benefits from combining “show” (visual generation) and “tell” (textual or latent reasoning). A future system could let visual generators present spatial states, deterministic tools verify them, and reasoning modules handle planning.

Paper: https://arxiv.org/abs/2607.21072

Project site: https://zju-omniai.github.io/ProVisE/

GitHub: https://github.com/ZJU-OmniAI/ProVisE