Redis Replication Evolution: From SYNC to PSYNC‑AOF and Shared Buffers

1. Redis Master‑Slave Replication Principles

Redis achieves high availability by keeping identical data copies on multiple machines connected via the network. The master sends data changes to its followers, typically using asynchronous replication.

2. Evolution of Redis Replication

2.1 SYNC (pre‑2.8)

Before version 2.8, Redis used a full‑sync SYNC process: the master forks a process, creates a snapshot (RDB), sends it to the replica, and then streams buffered commands. This approach suffers from slow fork, high copy‑on‑write memory usage, and the need for a full sync after disconnection.

2.2 PSYNC (Redis 2.8)

PSYNC introduces partial resynchronization. Replicas store a run‑id and offset; when reconnecting, the master can send only the missing command backlog if the offset is still present, avoiding a full sync.

2.3 PSYNC2 (Redis 4.0)

PSYNC2 adds support for failover scenarios by tracking a secondary replid (replid2) and its offset, allowing a new master to perform partial sync with existing replicas.

2.4 Disk‑less Replication (Redis 6.0)

Redis separates snapshot generation from disk I/O: the child process dumps data to a pipe, and the parent streams it directly to replicas, eliminating disk writes during full sync.

2.5 Shared Replication Buffer (Redis 7.0)

To reduce memory consumption with many replicas, a shared buffer is introduced. Each buffer block carries a reference count; replicas share blocks, and blocks are freed when the count reaches zero.

typedef struct replBufBlock {
    int refcount;
    long long id;
    long long repl_offset;
    size_t size, used;
    char buf[];
} replBufBlock;

3. Baidu Cloud Redis PSYNC‑AOF Solution

PSYNC‑AOF combines the AOF file with the replication backlog. When the backlog cannot satisfy a replica’s offset, the system searches the AOF for the missing data, enabling incremental sync without falling back to a full sync.

The AOF files are sent by a dedicated thread using send_file, avoiding blocking the main service thread.

4. Future Directions

Potential improvements include PSYNC3 (AOF‑based replication), SYNC‑less replication that skips full sync entirely, and multiplexed transmission of RDB, PING, and replication streams to further boost efficiency.

5. Summary

Redis replication has progressed from full‑sync SYNC to sophisticated partial‑sync mechanisms, disk‑less transfers, shared buffers, and AOF‑enhanced incremental sync, continually reducing latency, memory usage, and disruption during failover and high‑traffic periods.