Analysis of Intel Omni-Path vs. InfiniBand: Architecture, Products, and Performance

In earlier articles we analyzed InfiniBand technology, major vendors and products (mainly Mellanox); see the articles "InfiniBand Mainstream Vendors and Product Analysis" and "InfiniBand Technology and Protocol Architecture Analysis". The InfiniBand market is changing rapidly, and today only Mellanox and Intel remain active in this space.

The InfiniBand Trade Association (IBTA) has nine major board members—CRAY, Emulex, HP, IBM, Intel, Mellanox, Microsoft, Oracle, Qlogic—of which only Mellanox and Emulex focus exclusively on InfiniBand. Emulex was acquired by Avago in February 2015, and Qlogic sold its InfiniBand business to Intel in 2012.

Intel, after acquiring Qlogic’s InfiniBand business, launched a new high‑performance computing solution called “True Scale Fabric”, introducing the Omni‑Path Host Fabric Interface with 100 Gbps ports that directly challenge InfiniBand EDR.

Intel’s True Scale Fabric software architecture and product line constitute a complete network solution stack, which also implements the InfiniBand protocol stack; the Omni‑Path stack is built on top of this architecture. Below we discuss Intel True Scale Fabric’s software architecture and products.

The diagram shows the InfiniBand software stack implementation. Intel provides two programming interfaces at the upper layer: Verbs and PSM (Performance Scaled Messaging library). PSM is designed for MPI, offering a lightweight library optimized for MPI communication, while other applications can still use the Verbs interface. Intel HCA cards operate in On‑Load mode.

The next diagram illustrates Intel’s comprehensive True Scale InfiniBand product line. Given the demand for future high‑performance, high‑bandwidth applications, why develop Omni‑Path when InfiniBand already exists?

From a simple analysis of port count and network topology, Omni‑Path shows advantages in device count, latency, and quality‑of‑service; the physical layer optimizations are particularly significant compared with InfiniBand.

The diagram resembles the earlier Intel True Scale InfiniBand stack because Intel acquired QLogic’s InfiniBand division and Cray’s interconnect unit, inheriting QLogic’s InfiniPath products and rebranding them under the True Scale name.

Omni‑Path continues the True Scale branding, with the main change being an increase of physical‑layer speed from 40 Gbps to 100 Gbps. To remain compatible with more open systems, Omni‑Path is based on the open‑source OFED framework (also used by Mellanox) and its APIs are publicly available.

Intel has integrated Omni‑Path functionality directly into its CPUs, which improves communication efficiency but also ties the network more closely to the processor, limiting openness.

Through the acquisition of Cray’s interconnect division, Intel introduced a 1.5‑layer concept called the Link Transport Layer, built on Cray’s Aries technology to optimize low‑level data transport, providing reliable layer‑2 packet delivery, flow control, and single‑link management.

Test data shows that under sequential read/write I/O models, Omni‑Path’s latency advantage is not obvious; however, under random I/O and specific workloads, Omni‑Path demonstrates a significant advantage.

The Omni‑Path product line is comprehensive, including HFI (Host Fabric Interface) NICs, switches, software stacks, cables, and chips.

Intel Omni‑Path Fabric100 series comprises edge and core switches that form an end‑to‑end HPC solution; the edge switches provide 100 Gbps port bandwidth and low latency (details can be found on Intel’s official site).

Simple Summary: The competition between InfiniBand and Omni‑Path reflects typical post‑merger product overlap, yet both have legitimate positions. Omni‑Path offers lower latency thanks to Aries‑based physical‑ and network‑layer optimizations and CPU‑integrated support, giving it certain advantages over InfiniBand.

Furthermore, Intel’s Fabric Builders Program has created a strong ecosystem, and several large supercomputing centers are already deploying Omni‑Path networks.

From my personal viewpoint, Omni‑Path does not introduce breakthrough changes to the protocol architecture, leaving Intel’s InfiniBand portfolio in an awkward position. While Omni‑Path does not provide order‑of‑magnitude improvements in latency or bandwidth, Mellanox’s InfiniBand continues to evolve rapidly. I hope both InfiniBand and Omni‑Path can find suitable market niches and leverage their respective strengths.

Warm Tip: Please search for “ICT_Architect” or scan the QR code below to follow the public account and get more exciting content.

It is said that friends who like and share have already reached the pinnacle of life.