LabVLA: From Thinking to Doing—What AI Still Needs to Master Scientific Labs

Background and Motivation

Recent advances in AI for Science have excelled at cognitive tasks such as protein folding, literature understanding, and material discovery, but they still lack the ability to manipulate physical lab equipment like beakers, pipettes, and heaters. The gap is not due to insufficient robot capabilities, but because scientific protocols embed extensive implicit procedural knowledge that data‑driven methods struggle to capture.

LabVLA: A Generalizable Scientific Embodied‑Intelligence Paradigm

LabVLA, a joint effort by Zhejiang University and Shanghai AI Lab, injects Vision‑Language‑Action (VLA) pre‑training into scientific experiment scenarios. The model learns to map natural‑language experiment descriptions to cross‑task, cross‑environment manipulation strategies, moving beyond fixed workflow execution.

RoboGenesis Data Engine

To supply the required training signal, the team built RoboGenesis, a knowledge‑enhanced simulation platform that generates large‑scale, diverse lab scenes and corresponding action trajectories. The pipeline consists of three steps:

Experiment Space Construction : Text descriptions are turned into reference images, which are then 3D‑reconstructed and physically annotated to create reusable lab assets.

Workflow Generation : Natural‑language commands (e.g., “transfer liquid from beaker A to beaker B and heat”) are decomposed into atomic skills, instantiated on various robot platforms, and randomized across scene, lighting, and object configurations.

Structured Experience Consolidation : Generated trajectories are filtered for consistency, then annotated with task steps, object states, camera parameters, and spatial relations to form the LabEmbodied‑Data set, providing high‑quality supervision beyond simple demonstration videos.

Benchmark Results on LabUtopia

In the LabUtopia simulated benchmark covering six laboratory tasks (pick‑and‑place, button press, door opening, liquid transfer, heating, and transport), LabVLA achieved average success rates of 71.1% (in‑distribution) and 70.0% (out‑of‑distribution), outperforming prior baselines. Fine‑tuning other embodied models with LabEmbodied‑Data yielded comparable gains, indicating the data set’s broad utility.

Real‑World Robot Validation

LabVLA was deployed on a physical Franka arm and compared against DreamZero and π0.5 across four typical lab tasks (shaking, pouring, magnetic stirring, funnel insertion). Each task collected 50 trials with random perturbations in object pose and workspace clutter. LabVLA maintained >70% success in most settings and matched DreamZero overall, while surpassing it in challenging generalization scenarios (e.g., 80% success on out‑of‑distribution clean positions). The experiments revealed that liquid‑pouring is most sensitive to positional offsets, and multi‑step vessel manipulation stresses long‑horizon planning.

Analysis and Limitations

The work reframes laboratory operation from an experiential workflow to a learnable embodied problem, establishing a full pipeline from simulated data generation to real‑robot verification. However, the current system behaves more like a “technical assistant” than a fully autonomous scientist: it cannot design new experiments or adapt strategies dynamically. Remaining bottlenecks include the scarcity of high‑quality real lab data, diverse equipment standards, safety constraints, and limited transfer across heterogeneous experimental domains.

Future Directions

The authors plan to apply LabVLA to real scientific settings at Zhejiang University, Fudan University, and Jingtai Technology, targeting synthetic biology, drug discovery, and molecular materials. By open‑sourcing models, code, and data, they aim to lower entry barriers and accelerate progress toward truly general scientific embodied intelligence.