Running Distributed Reinforcement Learning with Isaac Lab’s Newton Engine and Rerun Visualizer on PAI

1. Launch DSW and Prepare the Environment

Start the Data Science Workspace (DSW) from the Notebook Gallery and select the provided Docker image, which runs on GPU instances that only have CUDA cores (no RT cores required). After the notebook launches, download the example code, run the initialization cell, and execute the one‑click test script to verify that the environment is correctly configured.

2. Use Rerun to Observe Newton Rendering

Isaac Lab now decouples the physics engine from the visualizer, allowing you to switch among omniverse, newton, and rerun via the --visualizer argument. The example below selects the Rerun visualizer:

# Example launch command
/workspace/isaaclab/isaaclab.sh -p ./Examples/IsaacLabNewton/rl_games/train.py \
    --task Isaac-Cartpole-RGB-Camera-Direct-v0 \
    --visualizer rerun \
    --headless

When rerun is chosen, a lightweight web‑based 3D viewer is embedded directly in the notebook, eliminating the need for an external VNC window. The visualizer can record and replay simulation data by setting keep_historical_data=True in RerunVisualizerCfg.

The first execution compiles the Warp kernels (a few seconds). After env.reset(), the Rerun window displays the cart‑pole environment, and you can interactively adjust the camera view. Running the task for 100 steps demonstrates the pendulum motion within the Rerun viewer.

3. Single‑GPU and Multi‑GPU Reinforcement Learning

3.1 Single‑GPU Training

Use the following Docker image for a single‑GPU run:

dsw-registry-vpc.cn-beijing.cr.aliyuncs.com/pai-training-algorithm/isaac-sim:isaaclab-newton-v2-1-20260210

# Set environment variable
export EXT_ROOT=/mnt/data/NB12
cd ${EXT_ROOT}/code/Pai-PhysxTrainTools
# Launch training script
/workspace/isaaclab/isaaclab.sh -p ./Examples/IsaacLabNewton/rl_games/train.py \
    --task Isaac-Cartpole-RGB-Camera-Direct-v0 \
    --headless \
    --num_envs 512 \
    --max_iterations 12000

rl_games : the RL library currently supported by the IsaacLab‑newton branch.

--headless : disables all visual output to speed up training; if no visualizer is specified, the default is headless.

--num_envs 512 : runs 512 parallel environments on the single GPU.

--max_iterations 12000 : total training steps; the task converges quickly, so this can be reduced.

3.2 Multi‑GPU Training

For distributed training across multiple GPUs, use the following command:

# Set environment variable
export EXT_ROOT=/mnt/data/NB12
cd ${EXT_ROOT}/code/Pai-PhysxTrainTools
# Launch distributed training script
/workspace/isaaclab/isaaclab.sh -p -m torch.distributed.run \
    --nproc_per_node=4 \
    --nnodes=1 \
    ./Examples/IsaacLabNewton/rl_games/train.py \
    --task Isaac-Cartpole-RGB-Camera-Direct-v0 \
    --headless \
    --distributed \
    --num_envs 512 \
    --max_iterations 12000

--nproc_per_node : number of GPU cards to use (4 in this example).

--nnodes : number of nodes; set to 1 for a single‑machine setup.

--distributed : enables distributed training.

--num_envs 512 : environments per GPU; total environments scale with the number of GPUs.

--max_iterations 12000 : same iteration count as the single‑GPU case.

4. Training Performance Comparison

TensorBoard can be used to compare logs from single‑ and multi‑GPU runs. Multi‑GPU training increases the total number of parallel environments, leading to higher frames‑per‑second (FPS) and more stable reward curves, as shown in the accompanying plots.

5. Summary

The PAI platform now fully supports Isaac Sim and the Isaac Lab toolchain, including the Newton physics engine and the Rerun visualizer. All relevant notebooks, scripts, and user guides are archived under the Pai-PhysxTrainTools/IsaacLab230 and IsaacLabNewton directories, covering both Isaac Sim‑based and Newton‑based reinforcement‑learning workflows.