How 58.com Achieves Read/Write High Availability with Dual‑Master and Redis Caching

Architecture Overview

Each service in the 58.com local‑city platform consists of two MySQL instances that replicate bidirectionally (master‑master) and a Redis cache. The two databases hold identical data, while Redis stores frequently accessed, non‑real‑time data.

Read and Write High Availability

Read path : The service layer always reads from the primary master database for data that must be up‑to‑date. For data that can tolerate slight staleness, reads are served from Redis, eliminating the need for MySQL slaves and avoiding replication latency.

Write path : Writes are performed on the primary master. A standby MySQL instance replicates bidirectionally but normally receives no traffic. Both master and standby are exposed through a virtual IP (VIP) that points to the master; on master failure the VIP is switched to the standby, providing write‑side high availability.

The standby consumes roughly 50 % of compute resources, but after SQL tuning the master’s CPU usage often stays below 10 %, making the idle capacity acceptable.

Primary‑Key Generation

Because the two masters can accept writes, the application must generate globally unique identifiers (e.g., UUIDs or a distributed transaction‑ID service) rather than relying on MySQL AUTO_INCREMENT, which would require careful step‑size management across many shards.

Scaling Procedure

Stop the bidirectional replication. At this point the two databases contain identical data.

Introduce a sharding strategy (hash‑based, range‑based, or a routing‑table lookup) to partition data across multiple master‑standby pairs.

For each new master, provision a corresponding standby database and configure bidirectional replication.

This method requires only a brief outage for the replication pause and can double the node count each iteration (2→4, 4→8, …). Scaling by a non‑power‑of‑two factor would require additional data‑migration logic not covered here.

Cache Consistency Strategy

Invalidate the relevant Redis key(s) before touching the database. This ensures that a failure after the database update does not leave stale data in the cache.

Execute the database update inside a transaction and commit.

Invalidate the same Redis key(s) again after the commit. A read that occurs between steps 1 and 2 could repopulate the cache with the old value, so a second invalidation guarantees freshness.

Assign a fixed TTL (time‑to‑live) to each cache entry so that any residual inconsistency expires automatically after a bounded period.

Step 1 can also serve as a health‑check for Redis; however, if Redis failures are handled elsewhere, the initial invalidation may be omitted.