Master‑Slave, Multi‑Master & Leaderless Replication: Strategies and Pitfalls

1. Master‑Slave Replication

Master‑slave replication improves database throughput, enables read‑write separation, provides backup, and enhances high availability. One master handles writes while multiple slaves serve reads; updates on the master are automatically copied to slaves.

Each slave has a single master.

Each slave must have a unique server ID.

A master can have many slaves.

Replication modes:

Full sync : the client waits for all slaves to receive and persist data before the commit returns.

Semi‑sync : the client waits until at least one slave acknowledges receipt.

Async : the client returns immediately after the master commits.

Group replication : a group of nodes must reach consensus (majority > N/2 + 1) before a transaction commits.

Failover steps:

Detect master failure.

Elect a new master.

Reconfigure the system to use the new master.

Potential issues during failover include asynchronous lag, data loss from the old master, split‑brain scenarios, and timeout configuration.

Real‑world incident: an outdated MySQL slave was promoted to master; its auto‑increment counter lag caused ID reuse, leading to key collisions with Redis and data leakage.

2. Replication Lag

Binary logs are transferred over the network, inevitably causing delay; reads from slaves may return stale data.

Slower hardware on the slave.

High load on the slave.

Large transactions.

Mitigation: for latency‑sensitive operations, force reads from the master, accepting higher master load and increased failure risk.

3. Multi‑Master Replication

Multi‑master replication removes the single‑master bottleneck, allowing writes to any node and improving fault tolerance across data centers.

Drawbacks include potential write conflicts.

Conflict‑resolution techniques:

CRDTs (conflict‑free replicated data types).

Bidirectional merge algorithms.

Version‑vector tracking, similar to Git three‑way merges.

Operational transformation used in collaborative editors like Google Docs.

Topology options (MySQL supports only type “a” by default):

Topology “a” – basic master‑slave.

Topology “b” – requires node identifiers to avoid loops.

Topology “c” – offers better fault tolerance but may suffer from uneven network latency causing log overwrites.

Full‑mesh topologies can lead to out‑of‑order writes; timestamps alone are insufficient, so version vectors are recommended.

4. Leaderless (Quorum) Replication

Clients can write to any replica; the system ensures durability through quorum reads/writes.

Quorum rule: if w + r > n, a read is guaranteed to intersect with at least one node that has the latest write.

Sloppy quorum may miss intersections.

Concurrent writes may lack a deterministic order.

Partial write successes can cause read‑after‑write inconsistencies.

Failed writes are not rolled back, so later reads may see newer values.

Typical configuration uses simple majority for both reads and writes, providing tolerance for up to n/2 node failures. Dynamo‑style databases prioritize eventual consistency; stronger guarantees require transactions and consensus protocols.

Concurrent‑write detection relies on version numbers; the “last write wins” (LWW) strategy discards older writes. Using immutable identifiers (e.g., UUIDs) avoids conflicts.

5. Data Partitioning

Combining replication with partitioning distributes data across nodes.

Partitioning methods:

Range partitioning (key‑interval): supports range queries but may lead to uneven distribution.

Hash partitioning : evenly distributes keys but does not support range queries.

Secondary indexes can be local (per‑partition) or global (spanning partitions). Local indexes reduce write amplification but increase query cost; global indexes enable efficient reads at the expense of write overhead.

Rebalancing strategies:

Modulo‑based: simple but costly when node count changes.

Fixed number of virtual partitions: assign many virtual shards to balance load.

Dynamic splitting/merging: automatically adjusts shard count to data size.

Node‑proportional partitioning: each node holds a fixed number of partitions.

Routing decisions rely on coordination services (e.g., ZooKeeper) or built‑in mechanisms (MongoDB’s config servers, Cassandra’s gossip protocol).