How Transformers Enable Instant 3D Scene Generation from a Few Photos

Overview

QuerySplat is a forward‑only 3D Gaussian Splatting (3DGS) technique that can synthesize an explorable 3D scene from a handful of casually taken photos in seconds. The method does not require camera pose estimation.

Technical Approach

Previous feed‑forward 3DGS models use a pixel‑aligned paradigm: each Gaussian point is bound to a pixel and its camera ray. Small mis‑alignments across views cause ghosting, floating artifacts, or many inefficient small Gaussians. The authors illustrate this with the analogy of multiple sketches of a vase where a single misdrawn outline creates duplicate contours.

Topos‑Lite replaces pixel‑alignment with a query‑based design. A set of learnable Queries act as scene‑level slots. Each Query aggregates information across different images and regions, then decodes a group of Gaussian primitives. This allows the Transformer’s query‑and‑attention mechanism to organise continuous 3‑D space instead of only 2‑D image tokens.

Usage Example

Given a few uncalibrated photos of an object (e.g., a tabletop item, a coffee shop interior, or a bedroom), the model performs a single forward pass and outputs a 3D Gaussian scene that can be rendered from arbitrary viewpoints. No manual camera pose input is needed.

Performance

On the DL3DV‑Evaluation benchmark, Topos‑Lite achieves the highest PSNR, SSIM, and LPIPS scores in 2‑view, 4‑view, and 12‑view settings, including when only two images are provided. Qualitative results show sharper object edges, clearer textures, and more stable cross‑view structure compared with prior methods.

Capacity Scaling Interface

During training the number of Queries can be increased from 1,024 to 8,192. Each Query decodes its own set of Gaussian primitives. Larger decoders, higher‑resolution appearance features, and stronger geometric backbones can be added independently, suggesting a “3D scaling law” where speed and quality need not be a trade‑off.

References

Project homepage: https://inspatio.github.io/querysplat/

GitHub repository (code and model weights): https://github.com/inspatio/querysplat

Technical report (arXiv): https://arxiv.org/abs/2608.01186