Why Dual‑RAID Beats Triple‑Replication in Distributed Storage

Distributed Storage Overview

Distributed storage is widely used in private and public clouds, server virtualization, hyper‑convergence, etc. It includes block, object, and file storage and is gradually replacing traditional disk arrays.

Main vendors: Huawei, H3C, Inspur, and innovators such as DaoEntropy, XSKY, SmartX.

Most vendors base on Ceph with a triple‑replication architecture; DaoEntropy uses a dual‑RAID architecture that combines local RAID with a distributed two‑copy strategy, resembling a distributed disk array.

Triple‑Replication Drawbacks

Triple‑replication stores three copies across the network. When an OSD fails, data is rebuilt on other OSDs according to CRUSH rules.

Its disadvantages include:

Consumes network bandwidth, affecting business data flow.

TCP/IP packet loss and retransmission cause network jitter, leading to OSD rebuild cascades.

Severe cases can cause service interruption or data loss.

“Slow‑disk” effect: each HDD acts as a single OSD with low random IOPS (~120), becoming a bottleneck; uneven load and disk defects exacerbate the issue, risking performance degradation and crashes.

Dual‑RAID Architecture

Dual‑RAID combines node‑internal RAID with a network two‑copy strategy. Nodes use log‑structured software‑defined storage, providing pooled storage, local redundancy, ARC cache, online compression, automatic fault detection, and virtual OSD (vOSD) services.

When a disk fails, local RAID repairs data without triggering network rebuild, keeping the vOSD state healthy.

Advantages over network‑wide repair:

Hardware‑failure isolation reduces stability issues.

Automatic business avoidance ensures zero‑impact repair.

Local repair incurs minimal latency and no network bandwidth consumption.

Each node can tolerate one or more disk failures, whereas triple‑replication tolerates at most two consecutive disk failures.

Dual‑RAID also supports online addition or replacement of NVMe SSDs, moving the working data set into a two‑level cache and achieving performance close to all‑flash systems.

Conclusion

Compared with triple‑replication, dual‑RAID offers superior performance, stability, and data reliability, with excellent hardware‑fault isolation that reduces operational difficulty and labor costs.