An Overview of Curve: High‑Performance Cloud‑Native Distributed Storage System

Introduction Curve is a high‑performance, easy‑to‑operate, cloud‑native open‑source distributed storage system (CNCF Sandbox). It can be integrated with OpenStack to provide high‑performance block storage, with Kubernetes to supply persistent volumes (RWO, RWX), and with PolarFS as the storage foundation for cloud‑native databases, supporting compute‑storage separation.

Distributed Storage Overview Distributed storage can be classified into three categories: object storage, file storage, and block storage. Object storage is accessed via PUT/GET APIs for items such as images or audio files. File storage is a distributed version of traditional file systems (e.g., Ext4, NTFS). Block storage corresponds to cloud disks.

The key design elements of distributed storage are large capacity disks, random‑write capability, data durability, and high availability.

The data placement can be either centralized or decentralized, and consistency is ensured via protocols such as Ceph’s strong consistency or Raft, which requires a majority of replicas to acknowledge updates.

Curve Architecture Curve provides both block storage (CurveBS) and file storage (CurveFS). The system allocates space per disk, dividing each disk into segments and each segment into chunks, reducing raw data overhead. A disk corresponds to a service component managed by a disk process. Curve adopts the Raft consensus algorithm because Ceph’s strong consistency caused latency issues during disk replacement or node failures in NetEase’s production environment.

To accelerate development, Curve builds on Baidu’s Braft and brpc libraries. For high‑performance I/O, Curve integrates SPDK and RDMA, and uses bcache for hybrid storage (NVMe/SSD for hot data, HDD for capacity). Issues such as RDMA sensitivity to network quality, missing block device letters after NVMe‑SPDK binding, and Brpc’s incomplete RDMA support were addressed.

Zero‑copy techniques and optimizations to the bcache write‑back mechanism were also implemented, along with handling of bcache garbage collection impact by separating WAL onto NVMe.

Main Highlights

1. High Performance Curve outperforms Ceph in random read/write, though sequential write performance is currently lower. Ongoing work focuses on improving sequential throughput, especially for kernel block services.

SPDK and RDMA have been deployed (2023 H1) to boost performance, with challenges such as network quality, monitoring of NVMe devices, and RDMA support in Brpc being actively resolved.

2. Easy Operations Curve can be deployed in minutes using documentation or the CurveAdm tool, which leverages containerization. An Operator‑style solution enables deployment in Kubernetes environments, though advanced features like automatic upgrades are still in progress.

3. High Stability Curve emphasizes stability through extensive unit, integration, and system testing, as well as hot‑upgrade capabilities. A centralized node monitors cluster load and health, while a chunk‑file pool reduces metadata overhead and supports snapshots stored in S3.

4. High Quality Every pull request triggers a CI pipeline that runs multiple test stages before merging, ensuring code quality and reliability.

RoadMap Future work includes expanding CurveFS (released June 2022) alongside CurveBS, adding AI‑specific optimizations, supporting Elasticsearch, and providing HDFS‑compatible interfaces. Enhancements for local snapshot support (currently S3‑only) are slated for Q4.

Q&A

Q: Does Curve’s write performance lag behind Ceph? Curve excels in random writes but lags in large sequential writes due to dual logging in Raft. Optimizations are underway to close this gap.

Q: What are the differences and connections between CurveFS and CurveBS? CurveFS is a file system, while CurveBS is a block storage system. They can interoperate: data can be stored either in S3 or in CurveBS, offering flexibility between performance and cost.

Q: When will SPDK and RDMA be open‑sourced? A version is already available on GitHub; final stabilization and documentation are expected by Q4.

Q: How does Curve integrate with Kubernetes? Curve provides the CurveAdm tool for container‑based deployment and an Operator for Kubernetes, simplifying installation and management.

Thank you for reading; the session concludes here.