Eliminating ‘Think‑Then‑Act’ Stalls: StreamingVLA Boosts VLA Speed by 2.4×

Systematic Analysis of VLA Latency

Vision‑Language‑Action (VLA) models follow a three‑stage serial pipeline—observation, action generation, and execution—causing frequent pauses that degrade interaction smoothness, especially on resource‑constrained edge devices. An analysis of the Pi0.5 model shows that the waiting time equals the sum of observation and generation times, which dominates the overall latency.

Design of StreamingVLA

1. Action‑Flow Matching (Parallel Generation & Execution)

Traditional VLA generates an entire action block via a diffusion process before any action can be executed. StreamingVLA replaces this with a state‑based flow where the model maintains an "action‑space state" that evolves over time. At each step the model predicts a velocity field for the state, integrates it to obtain the current action, and updates the state, allowing the newly generated action to be executed immediately while the next action is being predicted. To scale this to large VLA models, the authors extend the state to include physical trajectory alignment and modify normalization layers (removing offset terms and unifying scaling factors) to preserve additive properties.

2. Adaptive Early Observation (Parallel Observation & Execution)

After overlapping generation and execution, the remaining bottleneck is the observation phase. StreamingVLA introduces a lightweight Transformer‑based predictor that estimates the significance of pending actions by forecasting the change in image embeddings after those actions. If the predicted change is below a threshold, the next observation is started early, overlapping with ongoing execution; otherwise, observation waits until the action completes. This predictor adds only ~5% overhead to inference time.

Experimental Evaluation

Simulation (LIBERO Benchmark)

On four LIBERO tasks, StreamingVLA maintains a success rate of 94.9% (baseline 95.1%) while reducing per‑action latency from 49.9 ms to 31.6 ms (1.57× faster) and stall time from 230.8 ms to 36.0 ms (6.45× reduction), yielding a 2.4× end‑to‑end speedup.

Ablation Studies

Removing state alignment causes training failure, while its inclusion raises success to 97.1% and further cuts latency and stall. Replacing adaptive early observation with random early observation drops success from 94.9% to 90.9%, confirming the benefit of significance‑aware scheduling.

Real‑World Robot Test

Deployed on a Franka Panda arm for a pick‑and‑place task, StreamingVLA reduces average action delay from 271.49 ms (baseline Pi0.5) to 170.88 ms, a 1.58× improvement, demonstrating practical efficiency gains.

Conclusion and Outlook

StreamingVLA addresses high latency and pause issues in VLA deployment by parallelizing both generation‑execution and observation‑execution through action‑flow matching and adaptive early observation. The framework achieves substantial speed and smoothness improvements without sacrificing success rates, and its streaming execution concept may extend to other multi‑stage, multimodal real‑time systems.

Resources

Arxiv Link: https://arxiv.org/abs/2603.28565

Project Page: https://ghahahahag.github.io/StreamingVLA_Website/

Github Link: https://github.com/gen-robot/StramingVLA