Mastering Multi-Active Data Architecture: Reducing Write Latency and Ensuring High Availability

Foundations of Distributed Data Layers

Modern internet services adopt a three‑tier architecture (access, logic, data). The data layer must satisfy:

High availability : tolerate node, rack, data‑center or city failures.

High performance : support massive concurrent requests with low latency.

Scalability : grow capacity and functionality without redesign.

Cost efficiency : balance resource investment against business value.

Security : protect data privacy and prevent unauthorized access.

Multi‑functionality : accommodate evolving business requirements.

Multi‑Active (跨城多活) Deployment

When a service reaches city‑scale, fault tolerance becomes the primary goal. Fault domains are:

Single‑machine: early‑stage services, mitigated by backups and master‑slave replication.

Single‑data‑center: multiple rooms within a IDC to survive a data‑center outage.

Single‑city: services spread across distant cities to survive city‑wide disasters (typhoons, earthquakes).

Write Latency as the Bottleneck

Write latency grows sharply with geographic distance:

Intra‑rack round‑trip ≈ 0.5 ms.

Intra‑city (different rooms) ≈ 3 ms.

Inter‑city (e.g., Shenzhen‑Shanghai) ≈ 30 ms; longer for Beijing‑Shanghai or Shenzhen‑Tianjin.

At ~30 ms a single write incurs ~60 ms round‑trip; repeated cross‑city writes multiply this delay, often exceeding business latency budgets.

Two mitigation strategies are common:

Short‑distance synchronous replication : keep replication within 100‑200 km (e.g., Guangzhou‑Shenzhen) to achieve 5‑7 ms latency. This does not meet true cross‑city active‑active goals but reduces latency.

Asynchronous replication with near‑sharding : partition data by user geography so writes go to the nearest city; reads use local replicas. Cross‑city latency is avoided.

Sharding for High Write Volume

When a single node cannot sustain the write throughput, data must be split into multiple shards, each with its own write endpoint. For expanding, low‑access datasets (e.g., e‑commerce order logs), archiving or partitioned tables reduce storage pressure.

Isolation Sharding

Isolation prevents a failure in one shard from affecting others—"don’t put all eggs in one basket". Isolation is applied across the stack:

Access layer routes requests to the appropriate shard.

Logic layer processes only requests belonging to its shard.

Data layer can perform cross‑city synchronous replication per shard.

Read Path Considerations

Read latency is usually more user‑visible than write latency. Two read consistency models are used:

Read‑after‑write (strong consistency) : reads must see the latest write; effectively part of the write path.

Eventual consistency (delayed read) : stale reads are acceptable; reads can be served from the nearest replica.

Recommended practice is to place read replicas close to users (e.g., a Shanghai user reads from a Shanghai replica). When read volume is high, additional replicas are added. To avoid overloading the primary write node, cascading synchronization or caching layers can be introduced. A connection‑pooling proxy between the logic layer and the database limits the number of direct DB connections.

Data Replication Architectures

Beyond a simple primary‑secondary pair, several robust topologies are employed:

Three‑site five‑center : 1 primary + 4 replicas spread across three cities and five IDC rooms. Writes must be persisted to at least three instances to satisfy majority consensus.

Three‑site three‑center : minimal majority but higher operational risk; rarely chosen.

Same‑city three‑center : used when cross‑city latency is unacceptable; each city hosts its own three‑center cluster.

Dual‑master mutual replication : each instance holds the full dataset, but only one master processes a given write at a time to avoid conflicts.

Dual‑master replication relies on a global timestamp generator: Ti – insert timestamp. Ts – sync timestamp (when the record is replicated). Tb – ban timestamp (when a node is disabled after a failure).

If Ts < Ti < Tb, a write may be lost before replication, causing duplicate attempts. Data created before Tb but not yet synced must not be updated after failover to avoid conflicts.

Data‑Driven Routing Models

Based on data characteristics, three routing models are identified:

Global data : no sharding; reads are routed to the nearest replica, writes go to a single primary with multi‑city replication.

Near‑sharding data : data is sharded by geography; writes stay within the same city, reads use local replicas; cross‑city replication is asynchronous.

Cross‑city sharding data : shards are distributed across cities; writes may be synchronous across cities, but reads still prefer the nearest replica.

Architecture Selection Pattern

Designers should follow these steps:

Determine business scale and required fault‑tolerance level.

Quantify acceptable latency (e.g., <30 ms for cross‑city writes).

Assess cost constraints and resource availability.

Choose a replication model that meets the majority‑consensus requirement (e.g., three‑site five‑center for maximum resilience, same‑city three‑center when cross‑city latency is prohibitive).

Decide on sharding strategy (global, near‑sharding, or cross‑city sharding) based on data access patterns.

Plan read‑replica placement, caching, and connection‑pooling to satisfy read‑latency goals.

The final architecture balances performance, availability, resource utilization, and operational complexity.