Evolution and Fundamentals of Storage Systems: From Single-Node to Cloud‑Native

The article compiles information from Yang Chuanhui’s book "Principles and Architecture of Large‑Scale Distributed Storage Systems," the Dahuaxunhu series, various network resources, and the author’s own insights to provide beginners with a macro‑level understanding of storage development.

Storage has evolved from single‑machine solutions to internet‑scale infrastructures, continuously pursuing low cost, high performance, scalability, and ease of use; today it manifests as single‑node, centralized, distributed, cloud, and cloud‑native storage.

Regardless of the storage form, systems are built around specific data types, leading to distinct data classifications and storage data models.

Three common storage types are highlighted: block storage, file storage, and object storage.

Block storage relies on block‑level devices and includes Direct‑Attached Storage (DAS) and Storage Area Networks (SAN) as typical implementations.

File storage provides network‑based file services.

Object storage is built on key‑value stores, separates data and metadata, utilizes Object‑based Storage Devices (OSD), and offers RESTful APIs for access.

Single‑node storage encapsulates a storage engine that exposes file, key‑value, table, or relational models, supporting basic CRUD operations and varying engine capabilities.

Centralized storage expands on single‑node by adding components such as controllers, disk arrays (JBOD), front‑ and back‑end ports, and management devices, forming a more complex system architecture.

Distributed storage interconnects independent storage devices via a network to present a unified service, classified into distributed file systems, key‑value systems, table systems, and distributed databases, with design principles often referencing the CAP theorem.

Cloud storage builds on distributed storage, delivering storage services over the Internet with added flexibility, typically provided by cloud vendors.

Cloud‑native storage extends cloud storage to meet the dynamic requirements of cloud‑native ecosystems, supporting features like dynamic provisioning, S3‑compatible APIs, and Kubernetes integration.

Examples of cloud‑native storage projects include Rook, the first CNCF cloud‑native storage project that integrates file, block, and object storage into Kubernetes clusters, and MinIO, a high‑performance, software‑defined object storage suite that integrates seamlessly with Kubernetes.