LabVLA: Bridging AI Reasoning and Hands‑On Lab Automation

In the fast‑moving AI for Science field, large models excel at literature understanding, scientific reasoning and experiment planning, but converting abstract steps into stable, generalizable actions in a real lab remains difficult.

The gap is not due to insufficient robot capability; rather, laboratory work contains abundant implicit procedural knowledge that data‑driven methods struggle to capture, so most automation systems act only as “process executors”.

To address this, Zhejiang University and Shanghai AI Lab introduced LabVLA, a more generalizable scientific embodied‑intelligence paradigm that injects Vision‑Language‑Action (VLA) pre‑training into experimental settings, enabling models to learn cross‑task, cross‑environment manipulation from natural‑language descriptions.

Supporting this goal, the team built the knowledge‑enhanced simulation engine RoboGenesis and the LabEmbodied‑Data corpus. RoboGenesis creates scalable lab scenes by generating reference images from text, reconstructing 3‑D assets, and randomizing scene elements. It then decomposes natural‑language commands (e.g., “transfer liquid from beaker A to beaker B and heat”) into atomic skills, instantiates them on various robot platforms, and records structured trajectories with annotations such as object states and camera parameters, forming LabEmbodied‑Data.

LabVLA uses an open‑source large model as its vision‑language backbone and adds an action expert module that outputs continuous control signals. Training proceeds in two stages: a pre‑training phase on public robot datasets to learn discrete action token prediction, and a downstream phase that incorporates the action expert and fine‑tunes on LabUtopia‑style simulated lab data while applying a “knowledge‑isolation” mechanism to preserve visual‑language abilities.

On the LabUtopia benchmark, which covers six typical lab tasks (pick‑up, button press, door opening, liquid transfer, heating, transport), LabVLA achieves the highest average success rates of 71.1 % in‑distribution and 70.0 % out‑of‑distribution, surpassing existing baselines. Fine‑tuning other embodied models with LabEmbodied‑Data yields similar gains, demonstrating the dataset’s generality.

Real‑world validation on a Franka robot compares LabVLA with DreamZero and π0.5 across four tasks (shaking liquid, pouring, magnetic stirring, funnel insertion) under position perturbations and cluttered workspaces. LabVLA attains >70 % success in most settings and reaches 80 % success on clean, out‑of‑distribution positions, outperforming DreamZero in the most challenging scenarios. The experiments reveal that liquid‑pouring is most sensitive to positional and environmental noise, while multi‑step vessel manipulation stresses long‑horizon planning.

The authors argue that LabVLA transforms the historically informal “lab procedure” into a formalizable embodied‑learning problem, builds a complete pipeline from simulated data generation to real‑robot verification, and highlights current limits: diverse equipment, safety constraints, and costly high‑quality data still hinder fully general scientific embodied intelligence.

Future work will explore applications in synthetic biology, drug discovery, and molecular materials at Zhejiang University, Fudan University, and JingTai Technology, aiming to reduce human exposure to hazardous experiments and improve reproducibility.