ColaVLA Demonstrates Autonomous Driving Models Can Reason Without Text

In recent years the combination of autonomous driving and large models has become a hot research topic. A natural idea is to let a visual‑language model first understand the scene, then make a judgment, and finally output a trajectory, but most existing methods still perform reasoning as a chain of textual tokens, which introduces latency and a mismatch between discrete text and continuous control.

The paper from Tsinghua University and CUHK MMLab proposes ColaVLA, a framework that moves both reasoning and trajectory generation out of the textual domain. ColaVLA consists of two core components: the Cognitive Latent Reasoner , which performs high‑level driving cognition in a unified latent space, and the Hierarchical Parallel Planner , which expands the high‑level strategy into continuous trajectories.

The latent reasoner follows a four‑step process— Understand , Recognize , Rethink , Decide —all executed implicitly in the latent space. First, multi‑view visual inputs, driving hints and ego state are fed to a shared VLM to build a global scene understanding. Then an ego‑adaptive router selects the most relevant visual tokens (e.g., lane markings, nearby vehicles, traffic lights). Next, a set of learnable meta‑queries performs a second‑level “re‑think” on the compressed information, representing different high‑level driving strategies. Finally, the module outputs a high‑level driving prior directly for the planner, bypassing any intermediate natural‑language generation.

The Hierarchical Parallel Planner respects the inherent hierarchical nature of driving trajectories. It first determines a coarse‑grained intent and then progressively refines the details, preserving causal order. A causality‑preserving attention mechanism ensures information flows from coarse to fine scales without leakage. Crucially, the planner decodes multiple scales and modes in parallel within a single forward pass, avoiding the serial decoding required by text‑based chain‑of‑thought methods.

Experimental results on the nuScenes open‑loop benchmark show ColaVLA achieving the best composite performance among action‑based methods, with an average L2 error of 0.30 m and an average collision rate of 0.23 % , both improvements over the strong baseline SOLVE‑E2E. In the more demanding NeuroNCAP closed‑loop evaluation, ColaVLA attains an average score of 3.48 and reduces the collision rate to 36.8 % , outperforming several prior approaches, including the text‑reasoning ImpromptuVLA, even without explicit text chain generation.

Efficiency-wise, after engineering optimizations ColaVLA runs at 228 ms per frame on an H200 accelerator, which is roughly 5–10× faster than comparable text‑based methods. This demonstrates that moving reasoning to latent space yields concrete speed gains essential for real‑time autonomous driving.

Ablation studies confirm four key findings: (1) latent reasoning alone reduces trajectory error; adding the Rethink stage yields further gains, validating the “key‑capture then re‑check” cognitive chain; (2) the hierarchical parallel planner remains superior to plain MLP or diffusion heads even when the reasoning module is removed; (3) a balanced selection of critical tokens is crucial—too few loses information, too many adds redundancy; (4) generating trajectories hierarchically (key points first, then fine‑grained details) aligns with the causal structure of driving actions and outperforms single‑shot regression.

The authors conclude that autonomous‑driving inference does not need to be expressed as explicit text. By aligning the form of reasoning with the requirements of action generation—latent reasoning combined with a causally consistent, hierarchical parallel planner—systems can simultaneously improve safety, accuracy and real‑time performance, pointing to a promising direction for future large‑model autonomous driving research.