Why Ethernet Is Overtaking InfiniBand in AI and Data Center Networks

IDC reports that the global switch market reached 308 billion CNY in 2022, growing 17% year‑over‑year, with a projected CAGR of 4.6% from 2022 to 2027. China’s market was 59.1 billion CNY, up 9.5% YoY, and is expected to grow faster than the global rate, maintaining a 7‑9% annual increase.

Traditional data centers concentrate compute resources in a single location, whereas distributed computing spreads workloads across many interconnected servers. Large‑scale generative‑AI models such as ChatGPT, BERT, and DALL‑E require massive compute power, and high‑speed, low‑latency networks—especially switches—are essential to move data quickly between nodes.

Key switch performance indicators include overall switching bandwidth, latency, jitter, and compatibility. High bandwidth determines the total data exchange capacity, low latency is critical for real‑time applications, jitter measures timing variability, and compatibility ensures seamless integration with other network devices and protocols.

Ethernet switch chips consist of a switching core, an interface controller, and memory. The core handles packet preprocessing and forwarding, while a dedicated PCIe link connects the chip to the CPU, allowing the processor to issue forwarding commands.

According to the prospectus of Feiling Kesi, the switch chip accounts for about 40% of the direct material cost of a switch, making it the most expensive component. Other components include power supplies, chassis, PCBs, network transformers, and passive parts.

InfiniBand is a high‑performance interconnect standard designed for HPC clusters. Originating in 1999 from the merger of Future I/O and Next‑Generation I/O, it offers high bandwidth, low latency, and strong reliability, initially aiming to replace PCI, Ethernet, cluster interconnects, and Fibre Channel.

The rise of InfiniBand was driven by Mellanox, which now holds roughly 80% of the global InfiniBand market after acquiring several companies. A major advantage of InfiniBand is RDMA (Remote Direct Memory Access), which bypasses the CPU and kernel memory, reducing data‑transfer latency to around 1 µs and lowering CPU load.

Ethernet remains the most widely used networking technology and the foundation of the Internet. Since its inception, Ethernet has evolved from 10 Mbps to 400 GbE, supporting higher bit rates, more nodes, and longer links while maintaining backward compatibility. However, Ethernet lacks built‑in flow‑control, which can cause congestion under heavy loads.

RoCE (RDMA over Converged Ethernet) brings RDMA capabilities to Ethernet. RoCE v1 implements RDMA on Layer 2, while RoCE v2 encapsulates RDMA in UDP/IP on Layer 3, improving efficiency. RoCE combines the low‑latency benefits of InfiniBand with Ethernet’s ubiquity, making it suitable for data‑center environments that require zero packet loss and minimal delay.

When comparing IB and Ethernet, InfiniBand excels in bandwidth, latency, and reliability, making it ideal for high‑performance communication scenarios. Ethernet offers lower cost, broader compatibility, and easier integration with end‑devices, but may require more complex configuration for large‑scale, high‑throughput deployments.

Major vendors are shaping the future of networking: Nvidia is entering the Ethernet market, expecting a 20‑30% annual growth in its networking division; Intel withdrew from IB development in 2002 and now focuses on advanced Ethernet adapters; Broadcom dominates Ethernet switch chip production with products like Tomahawk and Trident; Huawei emphasizes Ethernet, releasing the CloudEngine series for AI‑heavy data centers; ZTE invests heavily in Ethernet, offering 800 Gb, 400 Gb, and 100 Gb solutions.

Industry forecasts predict that by 2028, 45% of generative‑AI traffic will run over Ethernet. The Linux Foundation’s Super Ethernet Alliance, founded in July 2023 by AMD, Arista, Broadcom, Cisco and others, aims to define and develop the Ultra‑Ethernet Transport (UET) protocol to meet the low‑latency, high‑scalability demands of HPC and AI workloads.