How Many Optical Modules Do A100, H100, and GH200 AI Clusters Really Need?

Traditional three‑tier data‑center designs have shifted to leaf‑spine architectures to accommodate the rapid growth of east‑west traffic, which accounted for over 70% of internal data‑center traffic in 2021 according to Cisco.

AI clusters demand high bandwidth, low latency, and lossless communication. Most large‑scale AI deployments use a Fat‑Tree network topology, and NVIDIA leverages NVLink for efficient GPU‑to‑GPU interconnects.

A100 Cluster Network and Optical‑Module Estimation

The DGX A100 SuperPOD consists of 140 servers (each with 8 GPUs) plus leaf‑spine switches (40‑port, 200 Gb/s per port). The three‑layer topology requires 1,120–1,124 cables per layer. Assuming copper cables with two 200 Gb/s optical modules per cable, the ratio of GPU : switch : optical module is 1 : 0.15 : 4; for a fully optical network the ratio becomes 1 : 0.15 : 6.

H100 Cluster

The DGX H100 SuperPOD contains 32 servers (8 GPUs each) and 12 switches, using an InfiniBand Fat‑Tree with 400 Gb/s ports that can be aggregated to 800 Gb/s. For a 4‑unit (4SU) cluster with a full‑optical leaf‑spine, each server‑to‑switch link uses a 400 Gb/s module, resulting in 256 GPU‑connected optical modules and 640 switch‑connected modules. The resulting ratio is approximately GPU : switch : optical = 1 : 0.08 : 1 : 2.5.

GH200 Cluster

The DGX GH200 SuperPOD integrates 256 GPUs linked via NVLink switches. Each NVLink switch provides 32 ports at 800 Gb/s, delivering 900 GB/s bi‑directional (450 GB/s uni‑directional) bandwidth per GPU. The total upstream bandwidth for the cluster reaches 115,200 GB/s. A fully optical implementation requires 2,304 × 800 Gb/s modules, giving a ratio of GPU : optical module of 1 : 9.

When scaling multiple GH200 clusters using the H100 architecture, the optical‑module demand varies: a three‑layer topology yields a GPU : optical ratio of 1 : 2.5, a two‑layer topology yields 1 : 1.5, and the combined upper bound reaches approximately 1 : 11.5.

Industry Examples

Meta’s Research SuperCluster deploys 2,000 A100 servers (16,000 GPUs) with 2,000 switches and 48,000 links in a CLOS leaf‑spine network. A full‑optical version would need about 96,000 × 200 Gb/s modules, matching the 1 : 6 ratio derived for A100.

AWS’s second‑generation EC2 Ultra Clusters (P5) combine H100 GPUs and custom Trainium ASICs, offering 3,200 Gbps aggregated network bandwidth and supporting up to 20,000 GPUs. The design uses 800 Gb/s optical modules, but currently does not require the higher‑capacity 800 Gb/s modules for all links.

Google’s latest TPU clusters employ a three‑dimensional torus topology. Each TPU connects to 2 neighbors in a 1‑D ring, 4 in a 2‑D plane, and 6 in 3‑D. Optical modules are required at a ratio of 1 : 1.5 per TPU, using DAC connections, wavelength‑division multiplexing, and circulators with 800 Gb/s VFR8 modules.

Conclusion

As AI compute clusters scale, the demand for high‑speed optical modules grows dramatically. Upgrading from A100 to H100 increases the required optical module capacity from 200 Gb/s to 800 Gb/s. GH200’s NVLink‑based architecture further amplifies the demand, reaching a 1 : 9 ratio per GPU, and multi‑cluster deployments can push this ratio to roughly 1 : 11.5, underscoring the critical need for advanced optical‑module technologies in future AI data centers.