Why NVMe‑over‑RoCE Is the Future of All‑Flash Data Center Networks

All‑flash data centers demand higher storage throughput and lower latency, prompting the emergence of the NVMe storage protocol. NVMe standardizes the SSD interface, leverages PCIe’s low‑latency parallelism, and simplifies the protocol stack, dramatically improving SSD read/write performance.

NVMe over Fabric (NoF) Options

NVMe over FC : Implements the FC‑NVMe standard on existing Fibre Channel networks, allowing FC‑SCSI and FC‑NVMe to coexist and reuse FC infrastructure.

NVMe over TCP : Uses the existing IP network and TCP transport to deliver end‑to‑end NVMe without changing the underlying network.

NVMe over RoCE : A form of NVMe‑over‑RDMA that runs on Ethernet‑based RoCE (RDMA over Converged Ethernet). RoCE, along with InfiniBand (IB) and iWARP, provides native RDMA capabilities.

Among these, Ethernet‑based RoCE offers higher bandwidth and lower latency than FC while retaining the IP‑based advantages of TCP, making it the dominant transport for NoF deployments.

Key Challenges for RoCE‑Based NoF

Network performance: Zero‑packet‑loss is essential; traditional Ethernet can suffer congestion‑induced loss.

Reliability: Sub‑second failover requires multiple network planes and rapid (<1 s) switchovers.

Ease of use: FC provides plug‑and‑play simplicity, whereas Ethernet‑based solutions need further adaptation for storage workloads.

Huawei NoF+ Enhancements

Network performance : NoF+ replaces static Ethernet water‑marking with predictive algorithms that adjust bandwidth proactively, achieving higher throughput and lower latency.

Reliability : The solution adds proactive fault detection and congestion notification, enabling sub‑second failover and maintaining system stability under high‑performance operation.

Ease of use : Integrated storage and networking products deliver a one‑click expansion and automated management, simplifying deployment and operation.

Data Center Network Architecture

Typical data‑center networking separates three planes:

Business network : Handles external service traffic.

Compute network : Connects OLTP/OLAP database servers, isolated from storage traffic.

Storage network : Dedicated physical network for server‑to‑storage communication, supporting zero‑RPO and near‑zero‑RTO disaster recovery.

For multi‑DC disaster recovery, two data centers are active simultaneously. If one DC fails, workloads switch to the other with minimal interruption, providing high availability and efficient resource utilization.

NVMe‑over‑RoCE in Multi‑DC Scenarios

Typical traffic patterns include:

Read operations from an application to storage nodes within the same DC.

Write operations that replicate data to a remote DC for backup.

Failover writes where the primary DC’s storage is unavailable, prompting the application to write to the secondary DC, which then synchronizes back to the primary.

RoCEv2 and Network Convergence

RoCEv2 moves RDMA onto Ethernet/IP, allowing the same IP network to support HPC, AI, distributed and centralized storage. This convergence enables a unified “three‑network‑in‑one” architecture where business, compute, and storage traffic all run over Ethernet, simplifying cabling and management.

In summary, NVMe‑over‑RoCE provides the optimal transport for NoF, delivering superior performance, reliability, and manageability, while Huawei’s NoF+ solution further refines these attributes to meet the rigorous demands of modern all‑flash data centers.