Design and Implementation of a Smooth Migration Scheme from bdrp to Baidu Redis 3.0

Abstract This paper designs and implements a smooth migration scheme for Baidu's internal use of Redis 3.0, aiming to improve the stability of the food‑delivery Redis service.

Background bdrp2.0, built on Redis 2.6 with some Redis 2.8 features, suffers from long MTTR (>10 minutes) during master‑slave switch because a full data sync is required. Existing solutions (Codis, Redis Cluster) cannot meet the requirements, while Baidu's Redis 3.0 solution provides high availability (SLA > 99.97%). Therefore the team chose Baidu Redis 3.0 and needed a migration path.

Comparison Analysis Three migration options were evaluated: loading AOF/RDB files (lowest cost but data loss of hours, unacceptable), using Redis AOF with an added socket write branch (seconds‑level persistence and real‑time sync), and the traditional RDB‑based master‑slave sync (more complex and higher development effort). The AOF‑based approach was selected, and its architecture diagram is shown.

Implementation Steps

For each master shard, add a new slave instance with AOF enabled; the new slave inherits the normal master‑slave sync behavior.

After sync, the modified instance performs AOF background rewrite, writing data to both the AOF file and a remote Redis socket.

Upon rewrite completion, the instance periodically persists new requests to the AOF file and socket, achieving real‑time sync after full sync.

Operations prepare consistency scripts to verify key counts and values.

If network jitter or rewrite failure occurs, the bgrewriteaof command can be manually triggered to restart the background rewrite.

Terminology

aof (Append Only File) : Redis persistence mechanism that appends write commands to a file.

aof buffer : Temporary buffer where write commands are stored before being flushed to the file.

aof rewrite : Process that compacts the AOF file by rewriting data as a minimal set of commands.

aof background rewrite : Forked child process performs the rewrite so the main process can continue serving requests.

aof rewrite buffer : Buffer that holds commands received during the rewrite, which are later merged into the new AOF file.

Sequence Diagram

The modified Redis AOF mechanism adds a step that writes each command to a remote socket in addition to the normal AOF flow, enabling loss‑less migration of bdrp commands to the Redis 3.0 cluster.

The three key points of the AOF mechanism are:

During background rewrite, Redis traverses the database and writes commands to a new AOF file.

While rewriting, the main process continues to accept requests, storing them in both the AOF buffer and rewrite buffer; after traversal, the rewrite buffer is flushed to the new file.

After rewrite, new requests go to the AOF buffer and are flushed to the file according to the configured policy.

By adding socket writes at these three points, the system achieves smooth, loss‑less data synchronization.

Summary and Outlook

The smooth migration scheme has been tested in offline environments, achieving real‑time synchronization and data consistency; it also paves the way for future upgrades to higher Redis versions. With Redis 4.0 now released, new features such as modules, incremental sync, LFU eviction, non‑blocking deletes, and memory optimizations can be considered for further enhancements.