Why Optical Interconnects Are the Next Bottleneck‑Breaker for Massive AI Clusters

1. Interconnect Demand and Bottlenecks in AI Clusters

Large‑model‑driven AI clusters require exponential growth in GPU count and bandwidth, while traditional copper interconnects improve only 1.4× every two years, creating a critical performance bottleneck.

AI model parameters grow ~400× every two years, demanding tens of thousands of GPUs and high‑frequency data exchange.

Super‑node architectures (32‑1000+ GPUs per node) need sub‑nanosecond latency, TB/s bandwidth, and >40 kW power per rack, which traditional interconnects cannot satisfy.

2. Optical Interconnect Technology Stack: Device‑Level to Chip‑Level

Optical interconnects are classified by the distance and integration level between the optical engine and xPU (GPU/NPU) into device‑level and chip‑level solutions, covering the full link from data‑center to on‑chip communication.

Device‑Level Optical Interconnect

Targeting cabinet‑to‑cabinet and cross‑cabinet scenarios, core technologies include optical switches and pluggable optical modules, supporting kilometer‑scale, high‑bandwidth links for clusters with thousands of GPUs.

Optical Switch (OCS)

Advantages: direct optical routing without O‑E‑O conversion, protocol‑agnostic 400 G‑1.6 T rates, <1 W per port power, nanosecond‑level latency, hundreds to thousands of ports.

Challenges: lack of packet‑level scheduling flexibility, reliance on centralized software orchestration, MEMS reliability and switching speed need improvement.

Pluggable Optical Modules

Mainstream products: 800 G modules now common; silicon photonics can reach 1.6 T, moving toward QSFP‑DD/OSFP high‑density packages.

Innovation: Linear‑drive optical modules (LPO) remove DSP, shifting signal processing to xPU SerDes, reducing power by 30 % and latency by 50 %.

Bottleneck: >1.6 T rates suffer signal integrity loss, requiring complex compensation algorithms that struggle to meet super‑node latency demands.

Chip‑Level Optical Interconnect

By shortening the optical‑electrical conversion distance, chip‑level solutions overcome electrical interconnect limits for intra‑node communication (centimeter‑to‑meter scale).

New Optical Interconnect Technologies

3. Core Technology Roadmaps for Chip‑Level Optical Interconnect

Silicon‑Photonic Integration (External Light Source + Silicon Engine)

Principle: CMOS‑based modulators (MZM/MRM) and detectors integrated on silicon, with external laser sources to avoid silicon’s low emission efficiency.

Modulator comparison:

MZM: wide wavelength range, good thermal stability, proven in pluggable modules for hundreds‑meter links.

MRM: 1/10 size of MZM, low drive voltage, 30 % power reduction, but temperature‑sensitive and requires precise control.

Advantages: high integration, CMOS compatibility, high bandwidth density with WDM, dominant route for CPO/OIO.

Challenges: external laser coupling loss (1‑2 dB per channel), need for ultra‑precise alignment, lack of unified standards.

VCSEL Solution (Vertical‑Cavity Surface‑Emitting Laser)

Principle: vertical emission directly modulated by current, eliminating separate modulators and enabling dense arrays.

Advantages: low cost, high energy efficiency for short‑range (tens of meters) links, mature in optical modules, suitable for NPO architectures.

Challenges: lattice mismatch between GaAs and silicon, low high‑temperature stability, >100 Gbit/s speeds need PAM4 optimization.

Micro‑LED Solution

Principle: GaN‑based micro‑LED arrays (hundreds per chip) combined via multi‑lane low‑speed aggregation to achieve high total bandwidth.

Advantages: Tbps/mm² bandwidth density, sub‑pJ/bit energy efficiency, ideal for <10 m intra‑rack links.

Challenges: insufficient stability above 100 Gbit/s, heterogeneous integration with silicon chips remains difficult.

4. Industry Status: International Leaders and Domestic Catch‑Up

International Industry

Standardization: OIF has released CPO framework, 3.2 T CPO modules, and external‑laser (ELSFP) standards.

Product deployment:

Broadcom – 2024 51.2 T CPO switch (8 × 100 Gbps ports), targeting 102.4 T by 2027.

NVIDIA – GTC 2025 showcased CPO switch with MRM modulators (200 Gbps per channel), 115.2 T total bandwidth.

Start‑ups: Ayar Labs (TeraPHY 4 T), Lightmatter (Passage L200 64 T), Avicena (Micro‑LED 1 Tb/s/mm).

Trend: CPO will migrate from switch side to compute side, with 800 G/1.6 T CPO expected in AI data centers by 2026‑2027, primarily for large‑model training.

Domestic Industry

Standardization start: CCITA released 1.6 T (switch side) and 400 G (NIC side) CPO specifications covering parallel and WDM routes.

Enterprise layout:

曦智科技 – first domestic xPU‑CPO co‑packaged prototype, planning a CPO switch chip by 2026.

奇点光子 – first‑generation optical I/O chip (32 × 6.4 T, <6 pJ/bit).

图灵量子 – LNOI photonic chip with GCS‑HiCPO packaging, supporting 102.4 T.

凌云光 – OCS matrix (320 × 320), 2.7 dB insertion loss, >188 billion port‑hours operation.

Shortcomings: key silicon‑photonic wafers, high‑density packaging substrates still imported; ecosystem lacks end‑to‑end coordination.

5. Challenges for Large‑Scale Deployment

Chip‑Level Optical Interconnect Core Challenges

Standardization gaps: no unified interface, fiber coupling, or thermal design standards, leading to fragmented solutions.

Packaging complexity: reliance on TSV, TGV introduces yield and reliability issues.

Device performance bottlenecks: silicon waveguide loss, 1‑2 dB coupling loss, temperature drift above 80 °C.

Thermal and simulation: local heat density >100 W/cm², multi‑physics simulation tools immature, lacking a common PDK.

Test verification difficulty: post‑packaged optical engines cannot be tested independently; high‑speed (800 G+) test equipment is costly.

Device‑Level Optical Interconnect Challenges

Optical switches lack O‑E‑O regeneration, have low loss tolerance, and require sophisticated software for dynamic topology.

LPO technology depends on xPU SerDes performance; cross‑vendor compatibility and standards are still evolving.

6. Future Roadmap and Call to Action

China Mobile proposes a three‑step “full‑optical supernode” architecture to accelerate large‑scale optical interconnect adoption, emphasizing that “optical‑in‑electric‑out” is inevitable as clusters expand to hundred‑thousand‑GPU scale, with silicon‑photonic integration as the mainstream direction.

Related Reading

NVLink vs. PCIe: GPU High‑Speed Interconnect Analysis

GPU High‑Speed Interconnect NVLink and PCIe Technologies

Blackwell GPU Architecture Evolution and Parameters