Alibaba Cloud's Panjiu Supernode: CPU-GPU Co-Evolution for Agentic AI at OCP 2026

At the 2026 OCP Global Summit in San Jose, Alibaba Cloud's server R&D team delivered two technical sessions outlining a full-stack hardware evolution for the Agentic AI era.

From GPU-Centric to CPU-GPU Co-Evolution

Dr. Chen Jian, Senior Engineering Director, described three paradigm shifts: early "CPU-centric," deep-learning "GPU-centric," and now Agentic AI's "CPU-GPU co-evolution." Agentic workloads — autonomous planning, tool use, long-context memory — explode token demand, pressuring three components:

Agentic CPU : must manage agent memory and sandbox over-subscription, requiring massive memory capacity.

AI Head CPU : handles inference pre/post-processing and emerging static-knowledge lookup (Engram/N-gram), also needing large memory.

Inference GPU : efficiency hinges on KV Caching; quantization helps but larger contexts still demand bigger KV Cache capacity.

Alibaba Cloud's answer is the UMX (Unified Memory Extension) architecture built on the Panjiu AL128 supernode cluster:

CXL switching and memory pools enable cabinet-level and cross-cabinet multi-tier memory expansion for Agentic CPU and head CPU.

UALink high-speed interconnect provides multi-tier KV Cache expansion for GPUs.

CPU, GPU, multi-tier memory, and interconnect operate in a unified architecture. As Chen summarized: "Agentic-era computing is collaborative… intelligence is built on the synergy of CPU, GPU, multi-tier memory, and high-speed interconnect."

Breaking the AI Memory Wall with Tiered Persistent Memory

Dr. Zhu Feng, Storage Architecture Director, framed the "AI memory wall": KV Cache now exceeds model weights in HBM consumption. In a 64-concurrency, 32K-context run of Qwen3-235B, weights occupy ~235 GB while KV Cache reaches ~384 GB (57% of HBM), and multi-round tool calls cause KV to accumulate continuously. "KV Cache has become the core contradiction determining inference cost and efficiency."

The solution is a multi-tier media system :

HBM/DRAM → hot data

SCM (Storage Class Memory) → warm KV

CXL SSD → inference hot path

TLC/QLC → training, checkpoint, cold data

Key hardware components:

AliSCM (custom non-volatile memory): CXL 2.0 Type-3, E3.S form factor, 256 GB–1 TB per module, byte-addressable, power-fail persistent. Measured read latency ~350 ns, read/write bandwidth 20 GB/s / 6.5 GB/s, 128-byte granularity, >2× bandwidth of previous-gen Optane.

AliMemory (CXL DRAM extension): ~140 ns latency.

AliFlash CXL SSD (XL-Flash): Gen7 target 1M IOPS 512-byte random read per drive.

These tiers are pooled via CXL switches and JBOM memory pools, aggregating capacity and IOPS across nodes.

Real-World Inference Validation

Under equal cost (128 GB AliSCM replacing 32 GB DRAM), SGLang + Qwen3-8B multi-turn dialogue tests showed:

Two-tier cache: throughput +58.3%, P90 TTFT -48.7%.

Three-tier cache: throughput +66.3%, P90 TTFT -36.0%.

Future Direction: UALink Scale-Up and ALink Convergence

With sparse attention becoming mainstream, HBM pressure may drop ~10× (only indices and active working set resident), but full KV still requires external tiered storage exhibiting small, random, serial access (~584 bytes/token/layer). Alibaba Cloud adopts UALink as the scale-up fabric (200 Gb/s per lane, <200 ns port latency, up to 1024 accelerators per pod, native load/store semantics). Next-gen AliSCM will connect directly to GPUs via UALink, with projected latency, cost, and power at 0.4×, 0.25×, and 0.125× of local DRAM respectively. Ultimately, these converge on the Panjiu UMX bus-level interconnect — a custom ALink fusing UALink and CXL to connect CPU, GPU, and multi-tier memory in a unified memory address space.

Product Portfolio Extension (Yunqi 2026 Alignment)

Panjiu AL128 supernode (new spec).

Panjiu AL64 supernode: scale-up to 1024 GPUs with SNPO all-optical interconnect.

Panjiu AL144 supernode: scale-up to 10,368 GPUs for training/inference.

Panjiu Agentic AI CPU Server: CPU, NIC, memory, SSD as peer resources composable via ALink Switch, natively meeting diverse Agent workload needs.

Full-chain KV Cache server storage solution: multi-tier, bidirectional flow, pooled sharing covering DDR5, AliSCM, CXL memory pools, AliFlash PCIe 6.0 TLC/QLC SSD, EBOF high-performance cache servers, with custom CIPU providing multi-tenant security isolation and high-performance access for AI training, inference, and Agent sandboxes.

Alibaba Cloud emphasizes that next-gen AI infrastructure standards cannot be set by a single vendor. Through OCP, CXL Consortium, UALink Consortium, and domestic ODCC, they will continue driving standardization of supernode compute, interconnect, and storage to build a more competitive, sustainable compute foundation for the Agentic AI era.