Understanding EMC vPlex Dual‑Active Storage Architecture and Distributed Cache Mechanisms

EMC's first‑generation dual‑active storage solution is built on the vPlex gateway product, which virtualizes heterogeneous storage from multiple vendors into a unified storage pool for servers. Since the 2010 release of vPlex, the hardware has progressed through three generations: VS1, VS2, and VS6. The VS6 hardware introduced in 2016 uses 40 Gb IB interconnects (replacing FC), delivering twice the performance of VS2, one‑third the latency, and supporting a larger number of volumes. vPlex 6.0 (VS6 hardware) also upgrades the OS to GeoSynchrony 6.0 with flash‑optimizations.

Its dual‑active capability is provided by the vPlex Metro Distributed Device (distributed mirrored volume) feature, enabling cross‑site active‑active applications. vPlex Metro consists of two vPlex clusters, each with its own local storage array; creating a distributed mirrored volume across the clusters provides dual‑active access, with identical volume IDs at both sites.

In the fabric, hosts connect simultaneously to both vPlex sites, while the two vPlex clusters interconnect via FC or 10 GbE; backend arrays connect only to their local vPlex. Both vPlex clusters and the vPlex Witness require IP network connectivity. The vPlex Management Console is a 1U server inside the VPLEX cabinet that provides a GUI management interface and, in dual‑engine or quad‑engine configurations, is backed by UPS power.

When external (including heterogeneous) storage LUNs are mapped to vPlex, they appear as Storage Volumes. Production Extents are created on these Storage Volumes (each Storage Volume can host up to 128 Extents, which cannot span multiple Storage Volumes). Extents are organized into Devices using RAID (supporting RAID 0, 1, and concatenation). Unused Devices become Top‑Level Devices; Virtual Volumes are created on Top‑Level Devices for host access and can be expanded online. Hosts access these Virtual Volumes through Storage Views, which are directly mapped to the hosts.

Starting with vPlex 5.0 (VS2 hardware), the vPlex Local and vPlex Metro product lines added vPlex Geo, extending the capabilities of vPlex 4.0 (VS1 hardware). vPlex Geo expands the Access Anywhere storage coverage, supporting up to 50 ms round‑trip latency between sites via asynchronous communication, enabling long‑distance data sharing, access, and relocation.

vPlex systems may contain one or two vPlex clusters. A vPlex Local system contains a single cluster, while vPlex Metro and vPlex Geo consist of two‑cluster systems. Each cluster provides Access Anywhere functionality. Every vPlex cluster includes a Management Console and can be configured with one, two, four, or eight engines, each engine having a backup power supply.

vPlex Local manages data movement and access within a single data‑center using a single cluster, supporting single, dual, or quad‑engine configurations (e.g., a local cluster with 4 engines and 8 controllers). Two local clusters can be combined into a Metro remote cluster with up to eight engines (16 controllers) forming a 16‑node AA cluster.

vPlex Metro, which uses two clusters, supports write‑through caching between the clusters; the clusters do not need to have the same number of engines (e.g., one cluster with two engines and another with four).

vPlex Geo enables Access Anywhere between two asynchronous, geographically distant nodes. Its distributed volumes use write‑back caching for Access Anywhere mirroring, and like Metro, the Geo system also comprises two clusters that need not have identical engine counts.

Both vPlex Metro and vPlex Geo can optionally include a Witness, which is deployed as a virtual machine (VMware‑only) in a third‑party site distinct from the two vPlex clusters. The Witness arbitrates between the clusters during site failures or communication interruptions, improving application availability.

The Witness works with vPlex Metro (RTO/RPO = 0) and vPlex Geo (RTO = 0, up to 50 ms inter‑cluster latency, RPO ≈ 0) to coordinate seamless failover, running as a VM outside the two clusters and connecting via IP.

The key capability of vPlex, Access Anywhere, is realized through Distributed Cache Coherence, ensuring data consistency both within a cluster and across remote clusters, enabling cross‑host, cross‑cluster, and cross‑data‑center access and synchronization. Controllers merge their local memory systems to form a distributed cache that can scale across vPlex Metro and Geo systems, providing global cache coherence.

Implementation of the distributed cache does not require every cache line to be identical; instead, a directory‑based approach tracks fine‑grained memory blocks with lock granularity to enhance scalability. Each engine’s cache is split into a 26 GB local cache and the remainder as global cache.

Read operations first check the local cache; if hit, data is returned immediately. If missed, the global cache is consulted; a hit there causes the data to be fetched into the local cache before returning to the host. If neither cache contains the data, it is read from the backend storage into the local cache, and both cache indices are updated.

Write operations in vPlex Metro use a write‑through approach: data is written to both sites, and only after both remote disk arrays acknowledge the write does the host receive a completion response. The cache coherence mechanism then updates cache metadata across all engines.

Because hosts at different sites can simultaneously read and write the same volume, the distributed cache coherence guarantees that all hosts always see the most recent consistent data.

During a write, the system first checks whether the data exists in the local or global cache. If not, it writes directly to the local cache; if old data exists, the old entry is invalidated before writing new data to the local cache, then to backend storage, and finally the global cache index is updated. In vPlex Metro and Local, writes complete after both clusters have persisted the data; in vPlex Geo, writes complete after the local cluster stores the data and the remote memory acknowledges.

Both vPlex Local and Metro employ a write‑through cache mode: when a virtual volume receives a write request, the data is written to the mapped backend storage volume; once the array confirms the write, vPlex acknowledges success to the host. This mode is not suitable for vPlex Geo, which uses a write‑back cache to accommodate larger RTT intervals; most applications cannot tolerate the latency of synchronous write‑through in such distances.

Conversely, vPlex Geo uses a write‑back cache: the host’s write is cached, mirrored to another local controller’s memory, and acknowledged to the host, after which the data is asynchronously flushed to the backend array. In the event of a power failure, any unwritten cache data is stored on a local SSD.

vPlex provides only storage virtualization and mirroring; features such as snapshots and replication require integration with RecoverPoint, so vPlex deployments often consider pairing with RecoverPoint.

Using vPlex for site‑to‑site dual‑active; vPlex writes host I/O to both data‑center storages and internally replicates I/O to RecoverPoint, which records each I/O for CDP‑based point‑in‑time recovery.

Implementing a 3‑DC solution: RecoverPoint at site 2 asynchronously replicates I/O to RecoverPoint at site 3, allowing both local and remote RecoverPoint appliances to capture every I/O for arbitrary‑time recovery.

vPlex Metro clusters support up to eight engines (16 controllers), read caching, and, when combined with RecoverPoint, CDP capabilities.

In summary, vPlex’s distributed cache accelerates read I/O and improves performance, but write I/O incurs additional latency because the system must query remote vPlex nodes for cached data, invalidate stale entries, and wait for remote confirmation before completing the write.

RecoverPoint’s recommended log‑volume configuration must account for extra capacity to store logs, as logs capture every change to production volumes; capacity planning must consider log growth, network bandwidth, stability, and storage performance.

When combined with RecoverPoint’s CDP solution, host writes can cause significant I/O amplification on the disaster‑recovery array. Due to these factors and the cost of dual‑active gateways, EMC introduced SRDF/Metro dual‑active functionality based on VMAX storage in 2015.

Warm Reminder: Please search “ICT_Architect” or scan the QR code below to follow the public account for more great content.

They say friends who like and share this post have reached the pinnacle of life.