Introducing Pi-zero: A General‑Purpose AI Foundation Model for Robotics

Physical Intelligence recently released π0 (Pi-zero), a general‑purpose AI foundation model for robots. Pi-zero is based on a pre‑trained vision‑language model (VLM) and outperforms other baseline models in evaluations on five robot tasks.

Pi-zero builds on the PaliGemma VLM and is further trained using a custom dataset collected from seven robots performing 68 tasks, together with the Open X‑Embodiment dataset. The resulting base model can accept natural language commands and execute tasks with basic proficiency. Researchers compared Pi-zero with two baselines, OpenVLA and Octo, across five tasks such as cloth folding and table tidying, and reported significant progress over the baselines.

The frontier of robot foundation‑model research includes long‑term reasoning and planning, autonomous self‑improvement, robustness, and safety. These directions are expected to advance markedly next year, painting a bright future for robot foundation models: highly capable general policies that inherit internet‑scale pretraining semantic understanding, integrate data from many tasks and platforms, and achieve unprecedented flexibility and physical ability.

Pi-zero's architecture is inspired by Transfusion, a Meta‑Waymo model that operates on tokens representing discrete and continuous data. Pi-zero adds a unique "action expert" module for robot‑specific action I/O. Its input combines visual images, robot joint angles, and language commands; its output is a sequence of robot action tokens.

For complex tasks, human language commands are first processed by a high‑level VLM that decomposes them into simpler subtasks, similar to SayCan. This approach improves performance on tasks like table setting, and giving the robot a series of simpler commands yields comparable gains.

Physical Intelligence co‑founder Karol Hausman confirmed on X that their demo videos were neither scripted nor remotely controlled. When asked why cloth folding was used for evaluation, he cited several reasons:

If done well, everyone can understand and use it.

Easy to reset (e.g., toss clothes back into a basket).

Can be of arbitrary length (continuous folding of multiple garments).

Easy to generate diverse data (many types of clothing).

A member of Andrew Ng's team likened π0 to GPT‑1 for robotics, signaling an emerging era of large robot foundation models despite the gap between abundant text data and scarce robot data.

Other major companies are also developing multimodal robot foundation models, including NVIDIA's GR00T (trained on video, text, and real robot demonstrations), Google's PaLM‑E (combining PaLM and Vision Transformer for robot control), and Google DeepMind's Robotics Transformer 2 (RT‑2), a visual‑language‑action model for robot control.

Related URL: https://www.physicalintelligence.company/

Related reading:

How Rust Will Change Robotics

Google Uses Robots to Delete Code: 20‑Year Accumulation of Billions of Lines, Deleting 5% C++ Code

10× Developers and AI: Will Elite Programmers Be Replaced?