Network Architecture and Performance Requirements for Training Large-Scale Generative AI Models

Training generative large models demands an ultra‑large‑scale, low‑latency, high‑bandwidth, highly available network substrate. The article studies the technical roadmap and implementation schemes for high‑performance network infrastructure, emphasizing workload‑aware custom network architecture design and transport protocol optimization for different training stages.

Key research directions include flow‑control and congestion‑control techniques, load‑balancing, automated operations, and deterministic network transmission for wide‑area RDMA.

1) Ultra‑large‑scale networking: Data parallelism, pipeline parallelism, and tensor parallelism coexist, requiring a "parameter‑wide network" spanning tens of thousands to millions of GPUs with massive capacity and bandwidth, while tensor parallelism operates within a single server.

2) Ultra‑high bandwidth: Intra‑node All‑Reduce traffic reaches hundreds of GB, and inter‑node GPU communication generates massive collective data, demanding per‑port bandwidth, abundant links, and bus technologies beyond PCIe.

3) Ultra‑low latency: Communication latency accounts for 20% of training time for hundred‑billion‑parameter models and up to 50% for trillion‑parameter models, exposing limitations of traditional flow‑control and ECMP load‑balancing.

4) Automated operations: As GPU clusters scale, network failures affect many GPUs; automated deployment, one‑click fault localization, and self‑healing are essential for stable computation.

Traditional three‑tier tree topologies lack scalability for massive model training; Fat‑Tree offers non‑blocking bandwidth but faces port‑count limits and high cost for tens of thousands of GPUs.

Current hierarchical network stacks use PCIe, NVLink, NVSwitch for intra‑node high‑bandwidth links, and NIC‑plus‑switch with RDMA for inter‑node communication, enabling any‑to‑any GPU data transfer across servers.

Most large‑model clusters reside in a single data‑center, but rapid growth in model, data, and compute scales pushes power, cooling, and space to limits, creating a need for long‑distance wide‑area interconnects. Wide‑area RDMA faces challenges in congestion, latency, and deployment.

Deterministic networking (Det‑Net) for wide‑area RDMA is emerging, with technologies such as FlexE, SPN, TSN, priority‑queue enhancements, and long‑distance congestion control promising to extend RDMA over long distances.