The article analyzes how the shift of data centers toward compute-centric architectures drives the need for high‑performance networking, explains the shortcomings of TCP/IP, compares InfiniBand, iWarp and RoCE protocols, and shows why loss‑less Ethernet with RDMA is critical for modern HPC workloads.
The 2022 AI‑2000 ranking highlights Alibaba Cloud as one of the ten most influential network research institutions worldwide, detailing its extensive publication record, breakthrough low‑latency RDMA technologies, NFC distance expansion, and the XLINK QUIC protocol that collectively reshape data‑center and wide‑area networking.
The article reviews high‑performance networking options—RDMA (including RoCE v2 and iWARP) and Linux’s io_uring—explaining their principles, hardware requirements, and benchmark results, and concludes that while RDMA delivers ultra‑low latency for specialized workloads, io_uring offers modest network benefits, leaving TCP as the default for most services.
This article provides a detailed overview of large‑scale RDMA technology, covering basic concepts, major protocols, network‑level techniques such as congestion control, lossless‑to‑lossy evolution and multipath, virtualization, communication libraries for AI training and storage, performance tuning, monitoring, and real‑world deployment experiences.
This article explains why traditional Linux kernel TCP stacks struggle with high‑performance demands, introduces shared‑memory IPC and RDMA concepts, describes the SMC‑R hybrid protocol that transparently replaces TCP sockets, and outlines practical acceleration methods and community contributions.
The article analyzes how the AI boom and high‑performance SSD storage demand sub‑microsecond latency, exposing TCP/IP’s inherent context‑switch and CPU overhead, and explains why RDMA’s kernel‑bypass, zero‑copy design and 1 µs latency make it the preferred network stack for modern data‑center workloads despite challenges in Ethernet deployment.
This article explains the fundamentals of Remote Direct Memory Access (RDMA), comparing it with traditional networking, outlining its core advantages, suitable use cases, the three main RDMA protocols (Infiniband, RoCE, iWARP), deployment requirements, communication flow, and essential programming concepts.
This article explains Intel Omni-Path Architecture and InfiniBand, describing their protocol stack, network topology, RDMA technology, and related protocols such as RoCE and iWARP, while also comparing their design goals and performance characteristics.
The article explains how high‑performance computing (HPC) networks overcome TCP/IP limitations by using RDMA‑based technologies such as RoCE v1/v2 and InfiniBand, detailing their architectures, advantages, vendor implementations, and cost‑effective migration to Ethernet‑based solutions for GPU‑driven workloads.
This article provides a comprehensive overview of Remote Direct Memory Access (RDMA), covering its definition, how it differs from traditional networking, core advantages such as zero‑copy and CPU offload, typical use cases, the three main RDMA protocols, deployment requirements, and essential programming concepts and terminology.
The article analyzes how the AI era’s demand for ultra‑low latency and high throughput exposes fundamental limits of the traditional TCP/IP stack, and explains why RDMA’s kernel‑bypass, zero‑copy design, and emerging congestion‑control algorithms are becoming the preferred network fabric for modern data‑center workloads.
This guide explains the fundamentals of InfiniBand and RDMA, details the InfiniBand Verbs API, outlines the steps required for kernel data handling, and provides practical configuration instructions for RoCE, IPoIB, and the subnet manager on Red Hat Enterprise Linux 8.
The article examines bus‑level data center network technologies, detailing how RDMA and ultra‑low‑latency forwarding mechanisms reduce end‑to‑end delays, enable high‑performance computing and AI workloads, and drive the evolution toward hyper‑converged, cloud‑native infrastructures.
The article analyzes how hyper‑converged data‑center networking unifies compute, storage, and HPC on a lossless Ethernet fabric, addresses AI‑era performance bottlenecks, compares RDMA over Ethernet with InfiniBand, and outlines the core metrics, value, and key technologies that enable zero‑loss, low‑latency, high‑throughput operation.
Alibaba Cloud’s self‑developed MRACC (Apasara Compute MapReduce Accelerator) leveraged hardware‑software integration, Spark and Hadoop optimizations, and eRDMA networking to achieve the top TPCx‑BB SF3000 performance, delivering up to 2‑3× faster SQL queries and 30% faster Spark shuffle, with significant cost efficiency gains.
The article explains how hyper‑converged data center networking, driven by AI‑era demands for zero‑packet loss, ultra‑low latency, and high throughput, replaces traditional separate networks with a unified Ethernet‑based solution that simplifies operations, reduces cost, and enhances performance.
This article explains the background, architecture, and performance of Alibaba Cloud's SMC‑R technology, which enables transparent, zero‑modification use of RDMA over standard socket interfaces, offering higher throughput and lower latency for cloud workloads while providing automatic fallback to TCP.
RDMA boosts data‑center performance by bypassing CPU‑intensive copies, but to realize its low‑latency promise you must build a lossless network using proper traffic classification—either Layer 2 PCP or Layer 3 DSCP—and combine PFC with ECN/DCQCN for precise congestion management.
The article explains how modern data‑center compute demands drive the need for high‑throughput, low‑latency networking, compares TCP/IP with RDMA‑based solutions such as InfiniBand, iWARP and RoCE, and recommends loss‑less Ethernet for large‑scale HPC deployments.
The article analyzes how AI‑driven workloads, exploding storage and compute capabilities, and distributed architectures expose the limits of traditional three‑network data‑center designs, and explains why a lossless, hyper‑converged Ethernet network with zero‑loss, low‑latency, high‑throughput characteristics is becoming essential.
This article explains the concepts of Direct Memory Access (DMA) and Remote Direct Memory Access (RDMA), compares traditional data transfer with DMA-enabled paths, outlines RDMA's advantages such as zero-copy and kernel bypass, and reviews the main RDMA protocols, standards bodies, and hardware ecosystem.
This article reviews Ceph's architecture, enumerates common benchmarking tools, analyzes its advantages and challenges, and presents a comprehensive set of performance‑optimization methods covering storage‑engine tuning, network communication, data placement, configuration parameters, hardware‑specific adaptations, and future research directions.
The article explains NVMe over Fabrics (NVMe‑oF), its evolution from PCIe‑based NVMe, the various transport mechanisms such as RDMA, Fibre Channel and TCP, and why networking is essential for high‑performance, shared, and disaggregated storage in modern data centers.
This article reviews Ceph's architecture, benchmarks, monitoring methods, and a wide range of performance‑optimizing strategies—including storage‑engine tweaks, network‑communication improvements, data‑placement algorithms, configuration tuning, and hardware‑specific adaptations—while also outlining future research directions.
This article explains how NVMe technology, combined with Intel's DPDK/SPDK user‑space framework, transforms enterprise storage by reducing I/O latency, simplifying the software stack, and enabling high‑performance NVMe‑oF, FCP, and iSCSI services with detailed architectural diagrams and implementation enhancements.
The article provides a comprehensive technical overview of InfiniBand, covering its history, standards, architecture layers, packet format, performance advantages, and a detailed comparison with Ethernet, highlighting why it has become the preferred high‑speed interconnect for HPC workloads.
This article explains Remote Direct Memory Access (RDMA), its origins in InfiniBand, the Ethernet‑based variants iWARP and RoCE (including RoCEv1 and RoCEv2), compares their architectures, performance characteristics, and deployment requirements for high‑performance computing and data‑center networks.
This article explains the fundamentals of RDMA and RoCE, compares RoCE v1 and v2, outlines deployment steps, highlights performance benefits such as low CPU usage and zero‑copy, and answers common questions about its differences from iWARP and InfiniBand, helping data‑center engineers evaluate the technology.
RoCE (RDMA over Converged Ethernet) enables lossless, high‑performance data transfer in data‑center networks by extending InfiniBand over Ethernet, with RoCEv1 operating at Layer 2, RoCEv2 adding UDP/IPv4/IPv6 routing, and Soft‑RoCE providing a software‑only solution for environments lacking RDMA‑capable hardware.
Tencent Cloud Block Storage (CBS) has evolved through three generations—apllo, atlas, and HiSTOR—adopting a client‑direct, distributed architecture, SPDK, RDMA and user‑space TCP to cut latency to sub‑microseconds while delivering exabyte‑scale throughput, high IOPS, and reliable multi‑copy replication for cloud VM workloads.
Tencent Cloud’s new CBS 3.0‑based Enhanced and Ultra‑Fast SSD cloud disks cut latency by over 50 % to sub‑100 µs, boost IOPS up to 1.1 million and throughput to 4 GB/s, and achieve these gains through SPDK‑driven virtualization, RDMA‑based data paths, a user‑space ZTCP stack, zero‑copy memory handling and dedicated hardware acceleration, targeting latency‑sensitive and IO‑intensive workloads such as large databases, video processing and AI inference.
This article provides a comprehensive overview of InfiniBand, covering its history, the InfiniBand Trade Association, architecture layers, packet format, switching mechanisms, performance advantages, and a detailed comparison with Ethernet for high‑performance computing environments.
The article explains how the fourth paradigm's distributed machine learning framework GDBT tackles the massive data, high‑dimensional features, and real‑time requirements of modern recommendation systems by leveraging heterogeneous computing, parameter servers, RDMA networking, and optimized workloads.
This article explains how Alibaba Cloud builds a high‑performance AI infrastructure by combining advanced hardware such as Shenlong servers, GPUs, FPGAs, NPUs, and custom interconnects like RDMA, together with virtualization, FPGA‑as‑a‑Service, AIACC, and resource‑pooling technologies to deliver scalable, cost‑effective AI services.
This article explains how InfiniBand’s RDMA‑based architecture, layered protocol stack, and Mellanox Socket Direct technology deliver far higher bandwidth, lower latency, and better CPU efficiency than traditional TCP/IP networks, and it presents performance test results that show up to an 80% latency reduction.
This article thoroughly explains the NVMe over Fabrics architecture, covering its encapsulation format, IO transmission steps, connection management, command extensions, discovery services, and the Linux reference implementation for RDMA and Fibre Channel transports.
This article explains how to design a low‑overhead, high‑performance lossless RoCE network for data centers, covering RDMA basics, mainstream network options, QoS, lossless and congestion‑control designs, buffer management, deadlock analysis, and practical tuning to achieve sub‑100 µs latency and near‑full bandwidth utilization.
This article provides a comprehensive overview of the OpenFabrics Enterprise Distribution (OFED) and the InfiniBand software architecture, covering its history, components, middleware, protocol stack, and how it enables high‑performance, low‑latency networking for IP, storage, and compute applications.
This article explains the NVMe over Fabrics architecture, compares various fabric transports such as FC, InfiniBand, RoCE v2, iWARP and TCP, and details how RDMA‑based technologies like zero‑copy, kernel bypass and CPU‑free transfers give NVMe‑oF its performance advantages while also covering protocol differences, FC‑NVMe, and the emergence of NVMe/TCP.
This article explains the OpenFabrics Enterprise Distribution (OFED) ecosystem, its history, the InfiniBand hardware and software stack, key protocols such as IPoIB, SDP and iSER, and how these technologies enable high‑performance, low‑latency networking across Linux, Windows and virtualized environments.
Alibaba engineers detail the PAISoar distributed training framework, showing how RDMA‑optimized hardware, Ring AllReduce algorithms, and user‑friendly APIs boost deep‑learning models—like the GreenNet CNN—to 101‑fold speedups on 128 GPUs, dramatically reducing training time from days to under a day.
This article provides a comprehensive technical overview of NVMe architecture, the NVMe‑over‑Fabric extensions—including InfiniBand, RoCE, iWARP, Fibre Channel, and TCP—explaining their RDMA‑based advantages, protocol differences, and practical considerations for data‑center storage deployments.
This article examines the rise of RDMA high‑performance networking, explains its technical advantages over traditional TCP/IP, showcases real‑world deployments in machine learning at Baidu, and explores future use cases in storage, GPU communication, and core services.
This article explains how RDMA (Remote Direct Memory Access) technology, originating from InfiniBand and extended to Ethernet (RoCE) and TCP/IP (iWARP), provides ultra‑low latency, high throughput, and minimal CPU usage for high‑performance computing and big‑data applications by bypassing traditional OS and protocol stack processing.
The article explains the evolution of NVMe‑oF, compares RDMA, FC and TCP transports, highlights the advantages and challenges of NVMe/TCP in modern data‑center and cloud storage, and discusses Lightbits' LightOS and accelerator card as a cost‑effective solution for high‑performance distributed storage.
This article provides an in‑depth exploration of the NVMe over Fabrics protocol, detailing its design goals, data encapsulation scheme, IO transmission process, command extensions such as Connect and Property Get/Set, and the associated Linux reference implementation for RDMA and Fibre Channel transports.
The article explains the emergence of the NVMe over Fabrics protocol, its performance advantages for SSDs, the role of RDMA and other transport layers, and how shared, cloud‑native storage architectures are reshaping enterprise IT infrastructure.
This article provides a comprehensive overview of Remote Direct Memory Access (RDMA), covering its background, limitations of traditional TCP/IP, related technologies such as TOE, U-Net, VIA, and detailed explanations of RDMA concepts, hardware implementations, verbs, and communication workflows.
This article explains the background of GPU communication, introduces DMA and RDMA fundamentals, describes how GPUDirect RDMA enables direct GPU-to-GPU memory access across machines, and presents performance results showing reduced latency and increased bandwidth for distributed deep‑learning training.
This article provides a comprehensive overview of InfiniBand technology, covering its open‑standard architecture, history, OFED software stack, protocol layers, performance advantages over traditional storage networks, and its primary use cases in high‑performance computing and data‑center environments.
This article explains the challenges of high‑performance computing and big‑data workloads on traditional TCP/IP stacks, introduces RDMA technology, its variants (InfiniBand, RoCE, iWARP), key protocols, hardware components, and how it achieves ultra‑low latency and high throughput with minimal CPU involvement.
This article examines the evolution from legacy SCSI and SAS storage protocols to NVMe and NVMe over Fabrics, explaining the performance bottlenecks of traditional storage, the technical advantages of NVMe, deployment options, vendor implementations, and future trends shaping data‑center storage architectures.
This article explains how Alibaba transformed its database infrastructure through hybrid‑cloud high‑performance ECS, container‑based multi‑instance deployment, and a user‑space storage‑compute separation architecture with RDMA, dramatically improving resource utilization, scaling speed, and cost efficiency for massive traffic spikes.
The article explains the background, architecture, and performance advantages of RDMA high‑performance networking, compares it with traditional TCP/IP, describes its deployment at Baidu for machine‑learning workloads, and outlines future use cases such as storage acceleration, GPU communication, and core services.
This article details Alibaba's journey from the 2017 pilot of storage‑compute separation to the 2018 large‑scale deployment of the DBFS user‑space file system, highlighting innovations such as zero‑copy I/O, RDMA integration, adaptive page cache, asynchronous I/O, atomic writes, online resize, and hardware‑software co‑design that enabled elastic, high‑performance database operations during the Double‑11 shopping festival.
The article examines how NVMe‑over‑Fabric extends NVMe beyond PCIe using various fabrics—FC, InfiniBand, RoCE v2, iWARP and TCP—highlighting RDMA’s zero‑copy, kernel‑bypass and CPU‑free advantages, and comparing protocol differences, performance trade‑offs, and the evolution toward NVMe/TCP.
The article describes Alibaba's HAIL 5.1 data‑center network architecture introduced for the 2018 Double‑11 event, detailing its high‑availability de‑stacking design, low‑latency RDMA deployment, and future HAIL 2.0 evolution to support larger‑scale, intelligent, and high‑performance cloud networking.
UCloud’s latest SSD cloud disk redesign dramatically improves performance—raising IOPS by 13‑fold, cutting latency tenfold, and expanding capacity—through a two‑layer IO path, 1 MB metadata shards, multithreaded models, overload protection, online migration, and upcoming RDMA/SPDK‑based ultra‑high‑performance storage solutions.
InfiniBand is a high‑performance network protocol that uses credit‑based flow control and switched fabric architecture to provide low latency, high bandwidth, and reliable data transfer, offering advantages over TCP/IP such as reduced packet loss, efficient RDMA, and support for various upper‑layer protocols.
This article provides a comprehensive overview of OpenFabrics Enterprise Distribution (OFED), its history, component stack, and the layered InfiniBand software architecture, explaining how various protocols such as IPoIB, SDP, and iSER enable high‑performance, low‑latency networking for Linux and Windows applications.
The article reviews Alibaba's large‑scale data‑center network advancements, covering its high‑performance evolution, programmable forwarding planes, massive RDMA deployment, automated control systems, AI‑driven self‑healing, and the Luoshen cloud network engine that underpins Alibaba Cloud services.
This article explains the fundamentals of NVMe‑oF, the role of InfiniBand components such as HCAs, switches, subnet managers and gateways, and why NetApp chose InfiniBand for its EF570/E5700 storage systems, highlighting performance benefits and protocol coexistence.
CynosDB is Tencent Cloud’s high‑performance, highly‑available NewSQL distributed database that uses a shared‑storage architecture with primary and replica instances, log‑structured storage, RDMA/SPDK transmission, Raft consensus, SSI MVCC, two‑phase locking, and state‑machine replication to provide elastic scaling, fast recovery, and cost‑effective enterprise data management.
Alibaba showcased its cutting‑edge network research at SIGCOMM 2018, highlighting programmable hardware, integrated software‑hardware designs, AI‑driven operations, the Hyper‑scale Edge Network vision, hierarchical network lifecycle management, large‑scale RDMA deployment, and advanced network visualization, positioning itself alongside leading global tech firms.
At SIGCOMM 2018 in Hungary, Alibaba showcased its pioneering network research, sharing visions on programmable hardware, integrated software‑hardware design, intelligent operations, the Hyper‑scale Edge Network, RDMA deployment, and network visualization, while presenting several influential papers and collaborating with leading academia.
The article provides a comprehensive overview of Remote Direct Memory Access (RDMA), explaining its underlying principles, performance advantages over traditional TCP/IP, various protocol families such as InfiniBand, RoCE, and iWARP, and practical implementation considerations for high‑performance networking.
DiDi PS is a custom RDMA‑based parameter server that uses a ring topology and optimized ibverbs communication to dramatically accelerate distributed deep‑learning training, consistently outperforming OpenMPI, NCCL2, TensorFlow’s built‑in RDMA, and Horovod while providing more stable and scalable synchronization for massive data workloads.
This article provides a comprehensive overview of Remote Direct Memory Access (RDMA), detailing its underlying principles, performance advantages over traditional TCP/IP, various protocol families such as InfiniBand, RoCE, and iWARP, and their respective hardware and software requirements.
The article provides a comprehensive technical analysis of NVMe over Fabric, highlighting the advantages of Fibre Channel over Ethernet, discussing RDMA, zero‑copy mechanisms, and the impact of SCSI translation layers on performance and adoption in modern data‑center storage architectures.
This article traces the evolution of relational databases, explains the rise of cloud‑native computing, and details how Alibaba Cloud’s PolarDB combines storage‑compute separation, RDMA networking, shared‑disk architecture, and advanced replication techniques to deliver high‑performance, scalable, and cost‑effective database services.
The article explains how RDMA (Remote Direct Memory Access) reduces CPU involvement, lowers latency, and increases bandwidth in data‑center networks, describes JD.com’s practical deployments across AI, big‑data, storage, and HPC workloads, and highlights industry trends toward broader RDMA adoption.
This article explains the growing need for more efficient data‑center networking, introduces Remote Direct Memory Access (RDMA) technology, describes its working principles, operation types, and how it can be layered over TCP/Ethernet to reduce latency and CPU overhead in high‑performance environments.
The article provides a comprehensive overview of RDMA technology, explaining its low‑latency, high‑throughput benefits, origins in InfiniBand, extensions like RoCE and iWARP, the Verbs API, communication models, and why it is poised for broader adoption in modern networking and storage systems.
Remote Direct Memory Access (RDMA) is a high‑performance networking technology that moves data directly between the memories of two computers without OS involvement, offering low latency, zero‑copy transfers and reduced CPU load through protocols such as InfiniBand, RoCE and iWARP.
This article provides a comprehensive overview of InfiniBand technology, covering its history, performance advantages over traditional interconnects, architectural concepts, layered protocol specifications, and typical use cases in high‑performance computing and data‑center environments.
