Why RoCE Is Overtaking InfiniBand in AI Compute: Insights from the UEC Alliance

UEC Alliance and Its Mission

The Ultra Ethernet Consortium (UEC), initiated by the Linux Foundation and its joint development foundation, aims to extend Ethernet beyond its traditional capabilities, delivering high‑performance, distributed, lossless transport layers for HPC and AI computing. Founding members include AMD, Arista, Broadcom, Cisco, Eviden, HPE, Intel, Meta, and Microsoft.

Recent UEC Developments

As of March 19, 2024, UEC added 45 new members and released the UEC Specification 1.0 whitepaper, outlining eight key functions and the performance benefits of Ultra Ethernet Transport (UET) for high‑speed data movement.

Broadcom’s RoCE Portfolio

Broadcom, a global leader in wired and wireless communications, leverages over 60 years of technology to offer more than 30 RoCE‑related products. Recent releases include the single‑port 400 GbE Ethernet adapter N1400GD and the 400 GPCIe NIC P1400GD, targeting AI, cloud computing, high‑performance computing, and storage networking.

Nvidia’s Shift Toward RoCE

Although Nvidia historically championed InfiniBand, it is now expanding its RoCE offerings. The company launched the Spectrum SN4000 and SN5000 switches and plans to introduce the 512‑port Spectrum UltraX800 in 2025 and the X1600 in 2026, delivering double the bandwidth of the current X800 models.

Meta’s RoCE‑Based Training Clusters

Since 2020, Meta has operated RoCE‑based distributed training clusters. Early consistency challenges were mitigated by deploying Arista 7800 and Wedge 400 switches, enabling 400 G interconnects that now power Llama 3 clusters.

RDMA vs. TCP/IP for AI Workloads

Remote Direct Memory Access (RDMA) aligns better with AI’s high‑concurrency, low‑latency requirements than traditional TCP/IP. RDMA enables the network system to access GPU memory directly via the NIC, bypassing the operating system and CPU, thus delivering higher throughput and lower latency for large‑scale parallel AI clusters.

Network Technology Comparison

InfiniBand : Designed specifically for RDMA, offering reliable hardware‑level transmission but at high cost and requiring dedicated IB NICs and switches.

RoCE : Built on Ethernet and UDP, consumes fewer resources, can operate over standard Ethernet switches, but still needs RoCE‑compatible NICs.

iWARP : Uses TCP for reliable transmission over Ethernet; while compatible with ordinary switches, it demands substantial memory for many TCP connections and requires iWARP‑specific NICs.

AI Compute Landscape: IB vs. RoCE

During the early AI compute boom, InfiniBand provided the optimal local solution, whereas RoCE now offers a broader, more cost‑effective optimum. Ethernet/RoCE’s mature ecosystem and lower deployment costs position it as the central fabric for future AI inference and training workloads.

Long‑Term Outlook

In the cloud computing domain, Ethernet/RoCE enjoys a deeper industrial foundation and lower costs compared to InfiniBand. As the technology matures and inference demand accelerates, Ethernet is expected to become the core networking substrate for AI compute.