Baidu Baige Adapts RLinf v0.4: LEGO-style Robot RL, 2.37x Throughput, Sim-to-Real

RLinf v0.4: Major Upgrades for Real-Robot Reinforcement Learning

RLinf, the first large-scale open-source RL framework for embodied intelligence, released v0.4 in October 2026 after four rapid iterations:

v0.1 (Dec 2025) : Established RL system abstractions, boosting development and runtime efficiency for complex RL tasks.

v0.2 (Apr 2026) : Unified robot hardware and GPU scheduling, enabling automatic discovery and flexible scheduling of heterogeneous robot resources.

v0.3 (Jul 2026) : Completed the full loop from data collection, management, supervised fine-tuning (SFT), RL, evaluation, to real-robot deployment, forming a unified dev loop from simulation to real-robot online learning.

v0.4 (Oct 2026) : Focused on real physical scenarios, systematically iterating around four pillars: LEGO-style real-robot system , high-efficiency real-robot algorithms , full-modality model support , and domestic compute foundation .

Project repository: https://github.com/RLinf/RLinf; documentation: https://rlinf.readthedocs.io/zh-cn/latest/index.html.

1. Modular Real-Robot Engineering (rlinf.robotics)

The new rlinf.robotics interface abstracts manipulators, dexterous hands, cameras, mobile bases, and teleoperation devices as plug-and-play "LEGO bricks." Hardware drivers and task logic are fully decoupled: changing robot bodies, reconfiguring, or deploying across nodes requires zero changes to upper-level task and policy code. Demonstrated end-to-end: dual-arm Franka with PICO VR teleoperation data collection, dual-arm HG-DAgger online training, and real-robot deployment. Added support for three new platforms: dual-arm YAM, AgileX Piper, and SO-101.

2. High-Efficiency Real-Robot Algorithms

Four new post-training algorithms tackle expensive real-robot samples, labeling difficulty, and inference latency:

RLT (Representation Decoupled Online RL)

STEAM (Zero-Label Self-Supervised Offline RL)

OPD (Online Policy Distillation): On LIBERO-Spatial, 30 training steps raised success rate from 67% to over 96% .

RTC (Asynchronous Real-Time Control): Cut inference wait time from seconds to ~2 ms .

3. Richer Model Ecosystem

Rewrote OpenPI (Pi0/Pi0.5) in native PyTorch, aligning operator-by-operator with the official JAX implementation, eliminating numerical precision gaps and convergence issues of community PyTorch ports.

Added five world/action models: NVIDIA Cosmos3, FastWAM, Pi0-FAST, Evo-1, MolmoAct2 .

Extended RL capability to image/video generation models SD3 and Wan2.2 for the first time.

4. More Realistic Simulation & Evaluation

RoboCasa365 : Large-scale household manipulation benchmark.

Real-World Simulator : Injects real sensor observation latency, cross-cluster and edge-cloud bandwidth limits into simulation, enabling high-fidelity regression validation before physical deployment.

New high-speed evaluation backends SGLang and ApxInf (in-process Rust/CUDA engine via APXinf-robo). Cosmos3 evaluation throughput improved 3.33x over direct integration.

5. Stronger System Foundation

Lossless observation image compression reduces network payload to 36%-41% of original size.

Upgraded collective communication tensor compression, weight reshuffling, and async weight sync.

Default runtime upgraded to PyTorch 2.11 with full-chain distributed tracing and profiling.

Added backends for Kunlunxin, Biren, Moore Threads, Intel XPU ; same training stack now covers NVIDIA, AMD, and six mainstream domestic/heterogeneous accelerators. Achieved up to 2.78x throughput improvement on Huawei Ascend and Moore Threads.

6. Ecosystem & Academic Recognition

RLinf accepted into PyTorch official ecosystem .

Became the official RL training engine integrated in NVIDIA Isaac Lab v3.0 .

RLinf team's WoVR (world-model RL) and RL-Co (sim-to-real collaborative RL) accepted at CoRL 2026 .

MoE communication fusion library FUSCO accepted at NSDI 2027 .

Baidu Baige: Three Ready-to-Run Scenarios for RLinf v0.4

RLinf v0.4 general image is live on Baidu Baige "Quick Start," pre-installed with 13 model environments, supporting multiple mainstream GPUs. One-click launch from the development machine eliminates environment packaging and deployment hassles.

Scenario 1: Cosmos3 Supervised Fine-Tuning on LIBERO

Base: NVIDIA Cosmos3-Nano world-action model. SFT on LIBERO action space yields a policy directly evaluable in LIBERO simulator. Official flow requires manual Cosmos3 code/dependency install and cosmos-framework setup; Baige pre-bakes these into the image. Developer steps: confirm environment → prepare base model & data → launch SFT training → convert checkpoint . Base model weights pulled from Baige internal storage; training monitored via TensorBoard, logs, and GPU metrics. Doc link: https://cloud.baidu.com/doc/AIHC/s/nmvhjh2z6

Scenario 2: Pi0.5 Evaluation on LIBERO via ApxInf

RLinf v0.4 adds SGLang and ApxInf high-speed evaluation backends. ApxInf runs model inference on GPU via APXinf-robo in-process Rust/CUDA engine; environment interaction and pre/post-processing stay in RLinf. Official approach needs manual CUDA binding compilation and install; Baige image pre-loads runtime and compiled artifacts. Activate environment and launch Pi0.5 eval on LIBERO-10 directly. Single-node 8-GPU runs 800 trajectories in ~12 minutes . Doc link: https://cloud.baidu.com/doc/AIHC/s/pmvhpkbg7

Scenario 3: Sim-to-Real Collaborative Training via RL-Co

Implements the RL-Co method (CoRL 2026): PPO in simulation + SFT on real-robot trajectories in the same optimization loop, boosting sim success rate while retaining real-data priors. Using ManiSkill digital twin and 50 real trajectories , two-phase collaborative training on Pi0.5. Baige single-node 8-GPU tested 100 steps: eval success rate rose from 50% to 75% , matching official results. Doc link: https://cloud.baidu.com/doc/AIHC/s/6mvhdmd19

Baidu Baige Full-Chain AI Infra for Embodied Intelligence

Embodied intelligence is in a pre-convergence phase with frequent architecture shifts. Open source drives algorithm innovation; AI Infra engineering determines whether models reach production at reasonable cost. Baige leverages HPC, extreme storage, and massive high-speed interconnect to build a complete workflow covering data prep, dev/training, sim verification, and inference deployment.

Data Pipeline

Operator-level optimization of egocentric video camera pose recovery pipeline: up to 43% reduction in GPU hours per video hour.

End-to-end DataLoader optimization for VLA and world-action models: FastWAM single-sample video processing time cut by over half , eliminating GPU data starvation.

Development & Training

Cosmos3: three optimization rounds (single-node tuning, 512-GPU scaling, heterogeneous pipeline) → MFU 0.42 , 64-node scaling efficiency 97.48% .

Precision-aligned PyTorch native OpenPI (via RLinf): Pi0.5 single-node 8-GPU throughput up 2.37x ; 256-GPU linear scaling ratio >91% .

Simulation Verification

Specialized optimizations for Isaac, MuJoCo ecosystems: Isaac Sim task startup time reduced 83.3% , Isaac Lab scenario FPS up 43.9% .

RLinf v0.3 adapted on release day; all experiments for the jointly released dVLA-RL framework (with Shanghai Jiao Tong University and Gua Robotics) ran on Baige.

Inference Deployment

Collaboration with SJTU Mu Yao team on AHA-WAM : single-step action inference latency compressed from 415 ms to 41 ms .