Understanding Data‑Control Separation Architecture in Modern Storage Systems

Data and control separation architecture, familiar from file systems such as Lustre, StorNext and BeeGFS, is also adopted by storage architectures (SDS), data‑service/ storage‑cloud software (e.g., Vipr), cloud computing platforms (e.g., OpenStack) and software‑defined networking (SDN).

Traditional file systems (NFS, CIFS) store data and metadata together on a single server (In‑band Mode), which becomes a bottleneck as client numbers grow, limiting throughput by CPU, disk I/O and network I/O.

For high‑growth data and extreme concurrency scenarios (e.g., HPC), distributed file systems (Isilon, OceanStor 9000) or high‑performance SAN file systems are required.

A modern high‑performance SAN architecture connects application servers directly to storage devices via a Storage Area Network, allowing only metadata to pass through a metadata server (Out‑of‑band Mode), thus improving data transfer efficiency and reducing metadata server load.

File systems such as CXFS, Lustre and BWFS use this structure, achieving better performance and scalability by separating control information from data transfer.

In high‑end storage, a typical design separates data and control caches; vendors like HDS and HPE implement this with separate Data Cache (user data) and Control Memory (metadata, system state, internal operational metadata).

Control Memory is stored both in the VSD’s local memory and on the DCA, with the DCA copy kept in a recoverable backup format, not a pure mirror.

Because metadata accesses are mostly internal to the VSD, the design behaves like a global‑plus‑private cache, providing fast metadata access and reducing DCA contention.

3PAR and OceanStor DJ also adopt data‑control separation, offering efficient handling of concurrent small transactional I/O and large bandwidth‑intensive I/O by physically isolating control and data paths.

Without separation, control and data traffic compete for shared memory buses and CPU resources, causing performance degradation.

OpenStack’s storage management follows the same principle: data‑control separation is used, with Cinder/Malina APIs exposing storage capabilities while VMs access data directly from the underlying devices.

ProphetStor’s Federator, an SDS platform, implements data‑control separation, providing unified storage discovery, abstraction, pooling and configuration, and integrates seamlessly with OpenStack components such as Cinder, Nova and Horizon.

Federator supports multiple back‑ends (NetApp, Nexenta, Ceph, FalconStor) and provides redundant data paths for applications.

On the control plane, Federator abstracts heterogeneous physical storage pools into a unified layer with REST APIs and a Web UI, enabling flexible storage provisioning.

OceanStor DJ offers similar capabilities, abstracting physical arrays into logical resource pools, delivering storage‑as‑a‑service (XaaS) and preserving native array features such as remote replication.

In summary, data‑control separation leverages bypass techniques to avoid changes to existing networks, prevents storage gateways from becoming performance bottlenecks, and improves concurrency handling, especially when memory bandwidth or CPU resources are limited.