Alibaba Panjiu AL128: 128-Chip AI Super Node Achieves Pb/s Bandwidth, Sub-150ns Latency

Alibaba Panjiu AL128 Super Node Architecture Overview

The Alibaba Panjiu AL128 super node server debuted at the 2025 Yunqi Conference and was updated with next-generation chips at the 2026 Alibaba Cloud Summit. It was selected as a "treasure of the hall" at the 2026 World Artificial Intelligence Conference (WAIC).

Current Generation Specifications

Integrates 128 AI chips in a tightly coupled configuration.

Single-cabinet bandwidth reaches Pb/s level.

Supports 64-GPU full-bandwidth interconnect.

Point-to-point (P2P) communication latency below 150 nanoseconds (sub-100ns level).

Evolution Roadmap

Alibaba Cloud has released the industry's first Scale-Up domain copper-optical compatible NPO module (internal code name SNPO) to prepare for the next-generation Panjiu super node, addressing higher bandwidth demands.

Multi-Dimensional Decoupled Open Architecture

The Panjiu AL128 adopts a multi-dimensionally decoupled open architecture that fully separates CPU, GPU, and ALink switch nodes. This design not only supports mainstream domestic and industry GPUs but also enables independent evolution of each component.

Key Components

Chip Foundation: Powered by Alibaba's self-developed T-Head Zhenwu M890 training-inference integrated AI chip.

Interconnect Hub: Uses self-developed ICN Switch 1.0 interconnect chip and ALink System interconnect protocol.

"Memory Wall" Solution (Panjiu UMX): Launched Panjiu UMX unified memory/storage expansion architecture. Via UALink and CXL buses, multiple storage media connect to compute nodes at nanosecond latency, solving memory bottlenecks at the system level.

Power and Cooling: Single cabinet supports up to 350 kW power delivery and 500 kW cooling capacity, using liquid cooling.

Modular Multi-Dimensional Decoupled Architecture: CPU nodes decoupled from GPU nodes, GPU nodes decoupled from ALink SW nodes, compute nodes decoupled from power nodes. This enables compatibility with mainstream CPUs, GPUs, and ALink SW chips, flexible support for independent evolution of primary chips, and flexible CPU-to-GPU ratios.

Panjiu AI Infra 2.0 AL128 Super Node Architecture Details

The super node integrates Alibaba's self-developed CIPU 2.0 chip and EIC/MOC high-performance NICs. The open architecture provides strong scalability, achieving up to Pb/s Scale-Up bandwidth and sub-100ns ultra-low latency. Compared to traditional architectures, inference performance improves by 50% at equivalent AI compute.

The full cabinet uses a custom dual-width rack design, supporting 128 GPU chips, 350 kW power, and 500 kW cooling.

CPU and GPU nodes are decoupled, GPU and ALink SW nodes decoupled, compute and power nodes decoupled. CPU and GPU nodes are laid out left-right separated, enabling flexible CPU-GPU pairing.

Interconnect uses a backplane-less orthogonal architecture: PCIe cables + copper cables + optical modules (secondary interconnect) directly connect Compute Trays (horizontal) and Switch Trays (vertical), reducing signal loss to achieve 112G SerDes high-speed connections between GPUs and ALink SW.

Liquid-cooled backplane cooling supports 500 kW heat dissipation with dual-loop CDU + cold plate design, achieving PUE ≤ 1.1.

Scale-Up Interconnect Topology

Each AI super node server contains 4 GPU chips. Each ALink SW node contains multiple ALink Switch chips, enabling 64–72 GPU chips in a single-switch topology with full-bandwidth, full-mesh interconnect within the Scale-Up domain.

Using a 64-port ALink Switch as an example: each ALink SW node houses 1–2 ALink Switch chips. The 64 GPU chips across 16 GPU nodes connect via orthogonal connectors: each GPU's Scale-Up port 0/8 connects to ALink SW0's switch chip; port 1/9 connects to ALink SW1's switch chip, and so on.

The upper and lower AL64-card super node interconnect domains support cross-expansion connections. However, under the single-switch Scale-Up principle, a 128-port ALink Switch chip is required.

Panjiu AI Infra 2.0 AL128 Super Node Parameters

The super node uses Zhenwu M890 chips with 144 GB VRAM per chip and 800 GB/s chip-to-chip interconnect bandwidth. A single cabinet integrates 128 AI chips (Pb/s-level bandwidth).

Reiterates: integrates CIPU 2.0 and EIC/MOC high-performance NICs, open architecture, Pb/s Scale-Up bandwidth, sub-100ns latency, 50% inference performance gain over traditional architecture at same AI compute.