Inside the Global AI Rack War: How GPU Supernodes Are Redefining Compute

What Is a Supernode?

Supernodes (or SuperPods) treat an entire rack as a single compute unit. In traditional AI clusters, scale‑up refers to intra‑server or intra‑rack interconnects (NVLink, PCIe) that provide high bandwidth and low latency, while scale‑out uses InfiniBand or Ethernet between servers or racks, offering lower bandwidth and higher latency. When a model exceeds the capacity of an 8‑GPU node—e.g., MoE models that need tensor parallelism across a ninth GPU—traffic must cross the scale‑out network, causing immediate bandwidth and latency degradation.

Supernodes extend the scale‑up domain from 8 GPUs to 72, 384, or even 1024 GPUs, presenting a unified address space and shared memory pool so that dozens or hundreds of GPUs can be scheduled as if they were a single card.

As Jensen Huang (NVIDIA) put it, “the rack is the unit of compute.”

Why the Industry Is Shifting to Supernodes

MoE all‑to‑all communication: After DeepSeek, most large models adopt mixture‑of‑experts (MoE) architectures. Inference routes each token to several experts spread across many GPUs, requiring all‑to‑all data exchange that is extremely bandwidth‑ and latency‑sensitive. An 8‑GPU domain cannot host enough experts, forcing traffic onto the slower scale‑out network.

KV‑cache demands: Longer context windows and higher concurrency inflate KV‑cache size. Supernodes provide tens of terabytes of uniformly addressable memory, enabling a distributed KV‑cache that 8‑GPU nodes cannot support.

Single‑card performance plateau: Process‑node scaling, HBM density, and power limits are approaching physical limits; single‑GPU performance gains have slowed, making cluster‑level interconnect efficiency the primary lever for cost‑effective scaling.

Commercial logic: Packaging GPUs, CPUs, interconnects, power, cooling, and software into a standardized rack turns the “cabinet” into a sellable product. Vendors shift from selling individual cards to selling complete compute units, while customers avoid the engineering effort of building their own clusters.

NVIDIA’s Roadmap

2024 GTC introduced the GB200 NVL72 (72 B200 GPUs + 36 Grace CPUs) with NVLink 5 and nine NVSwitches, delivering 1.8 TB/s per GPU and 130 TB/s aggregate bandwidth—considered the first true supernode. Subsequent generations include:

GB300 NVL72 (2025) – Blackwell Ultra B300 GPUs, 288 GB HBM3e per GPU, 20 TB total HBM, targeting inference and test‑time scaling.

Vera Rubin NVL72 (H2 2026) – Dual‑die GPU with 336 billion transistors, 288 GB HBM4 per GPU, NVLink 6 (3.6 TB/s per GPU), 190‑230 kW power, 45° water‑cooled liquid.

Future plans: NVL576 (576 GPUs, optical interconnect) and NVL1152 (1152 GPUs, direct optical links) slated for 2027‑2028, though a reported PCB issue for the Kyber cabinet may delay the latter.

Orders disclosed at GTC exceed $500 billion for Blackwell + Rubin, and deployments such as NVIDIA’s Stargate Abilene campus (45 million GPUs by 2025) and xAI’s Colossus 2 (1 million GPUs) illustrate the scale.

Huawei’s Approach

Huawei pursues a system‑level strategy, acknowledging its 7 nm process limitation. As Xu Zhijun said, “AI compute is not equal to chip performance.” Huawei’s four‑pronged tactic—system redundancy, space‑for‑compute, bandwidth‑for‑compute, energy‑for‑compute—leads to the CloudMatrix 384 (2025) with 384 Ascend 910C GPUs and 192 Kunpeng CPUs, interconnected by a self‑developed MatrixLink bus that provides full‑mesh memory‑semantic connectivity.

SemiAnalysis benchmarked CloudMatrix 384, noting a 4× power increase and ~2.4× per‑unit cost compared with competing solutions, yet confirming superior system capability. By mid‑2025, Huawei announced the Atlas 950 SuperPoD (8192 950DT GPUs, 160 racks, 8 EFLOPS FP8) built on the Lingqu 2.0 optical fabric and immersion cooling.

Wallin (壁仞) and Other Domestic Players

At WAIC 2024, Wallin showcased the NPO optical‑interconnect, a decoupled architecture supporting up to 1024 GPUs per supernode. The design addresses the “physical ceiling” of copper interconnects—bandwidth demands >1 TB/s cannot be met beyond ~3 m copper runs. Wallin’s solution includes:

BR2xx series GPUs (chiplet‑based, FP8/FP4 support) with native supernode interconnect.

BLink 2.0 protocol that enables memory‑semantic sharing across 1024 GPUs and offloads arithmetic to the switch (in‑network computing), with intelligent congestion control and multi‑hop self‑healing.

A three‑tier product matrix: 16‑GPU standard servers (electrical), 128‑GPU high‑density cabinets (electrical), and 1024‑GPU distributed decoupled supernodes (optical).

Other Chinese vendors presented comparable solutions:

沐曦 S600 – 64‑GPU rack with OEX orthogonal “zero‑cable” design, scalable to ten‑thousand‑GPU clusters.

燧原 ESL64‑O – 64‑GPU all‑optical rack using all‑domestic components, claiming ~80 % cost reduction.

中科曙光 scaleX640 – Record‑setting 640‑GPU rack, part of a 10 240‑GPU cluster delivering >5 EFLOPS.

摩线程 MTT C256 – 256‑GPU single‑layer scale‑up prototype, touted as breaking the 64‑GPU norm.

寒武纪 思元 690 – 256‑GPU supernode deployed in multiple AI centers.

平头哥 真武 810E – 96 GB HBM2e, 2.7 TB/s bandwidth, 400 W per card, positioned against NVIDIA H20/A800.

Design Philosophy Differences

Scale‑up protocol openness: NVIDIA uses proprietary NVLink, Huawei adopts the open UnifiedBus (Lingqu), AMD promotes UALink over Ethernet, Wallin’s BLink claims openness.

Interconnect medium: Copper remains dominant, but optical is the clear future. Huawei’s CM384 employs 6912 × 400 G optical modules; Wallin’s NPO is already in production; NVIDIA’s CPO optical switch is in development.

Ecosystem: NVIDIA’s CUDA ecosystem is deeply entrenched; Huawei open‑sources CANN; AMD advances ROCm; domestic players balance open standards with proprietary extensions.

Gap Analysis and Market Outlook

Three layers of gaps emerge:

Chip level: Domestic GPUs sit roughly two generations behind NVIDIA’s B300, mainly due to process node and HBM density. Roadmaps suggest convergence by 2028.

System level: Chinese 64‑/128‑GPU supernodes already match or exceed NVIDIA in many metrics; remaining gaps involve switch‑chip maturity, large‑scale stability, and power‑efficiency. Industry expects parity by 2027‑2028.

Ecosystem level: CUDA’s 20‑year legacy gives it a massive code base. New models (DeepSeek, Qwen, Kimi) are being ported to Ascend and Zhaoxin, but legacy code will linger for years.

华泰证券 estimates the domestic supernode market could reach ¥3.4 trillion by 2028, with a CAGR of 194 % from 2026‑2028.

Three practical procurement tips are offered:

Look beyond TFLOPS; compare measured MFU within the scale‑up domain, as identical EFLOPS machines can differ by up to 2× in MoE inference throughput.

Account for power and cost per effective compute unit—supernodes consume far more electricity than single‑node systems.

Clarify software‑stack responsibility; hardware vendors that also provide the full stack (model adaptation, fault recovery, scheduling) reduce integration risk.

Within three years, domestic supernodes are expected to be competitive for inference and medium‑scale training, while the frontier of large‑scale training will still be led by NVIDIA, albeit with a shifted competitive landscape.

Conclusion

The supernode era marks a shift from “single‑GPU” competition to “rack‑as‑the‑compute‑unit” competition. For NVIDIA, this extends its moat from silicon to the entire data‑center; for Chinese vendors, it offers a viable detour around single‑chip limitations by leveraging system‑level advantages, albeit with higher power, cost, and ecosystem challenges.