How to Build a 10,000‑GPU Supercluster: Core Design Principles and Architecture

Background

Currently, building clusters with more than ten thousand GPUs is still in its infancy, relying heavily on Nvidia GPUs and associated equipment. While domestic AI chips have made rapid progress thanks to policy support and application demand, they still lag behind in overall performance and ecosystem maturity, making the construction of a high‑performance, domestically‑sourced super‑cluster a significant challenge.

Core Design Principles

Extreme compute power : Use Scale‑up interconnects to achieve peak per‑node performance and Scale‑out interconnects to expand the cluster beyond ten thousand GPUs, combining both to form a massive compute base.

Collaborative optimization system : Leverage distributed training strategies such as DP, PP, TP, EP to continuously improve effective compute, maximize the compute‑communication ratio, and boost model development efficiency.

Stable long‑running training : Implement automatic detection and repair of hardware/software faults, increase MTBF, reduce MTTR, enable checkpoint‑resume capabilities, and support sustained training of trillion‑parameter sparse models over hundreds of days.

Flexible compute provisioning : Provide elastic scheduling, isolated resource allocation, and on‑demand distribution of training and inference workloads to maintain consistent performance across multi‑tenant, multi‑task scenarios.

Green low‑carbon operation : Deploy full‑stack liquid‑cooling solutions, pursue the highest FLOPs/W efficiency, and keep liquid‑cooling PUE below 1.10.

Overall Architecture Design

The super‑large GPU cluster is organized into four technical layers plus one operational domain (see diagram below).

1. Data‑center Support Layer

Accommodates the high‑density construction mode of a ten‑thousand‑GPU cluster, focusing on efficient power delivery, cooling design, floor load capacity, and cable routing.

2. Infrastructure Layer

Integrates compute, network, and storage resources. CPUs, GPUs, and DPUs cooperate to maximize compute; a multi‑plane RoCE‑based CLOS network provides high‑bandwidth, low‑latency connectivity with load balancing and isolation; converged and tiered storage enable non‑blocking concurrent data access.

3. Intelligent Computing Platform Layer

Built on Kubernetes, offering bare‑metal and container resources. It adds automated fault management for large‑scale clusters and prepares for heterogeneous GPU integration by introducing native compute abstractions to avoid platform fragmentation.

4. Application Enablement Layer

Consists of model‑training frameworks and development toolsets. It supports distributed training optimization, operator fusion, and communication‑computation overlap, while evolving toward automated model‑development pipelines.

5. Operations & Maintenance Domain

Provides efficient collective communication, flexible tenant‑based resource allocation, and multi‑task scheduling to ensure stable, high‑throughput training across the entire super‑cluster.