Redis Persistence Evolution and Baidu Cloud’s Forkless & Multi‑AOF Optimizations

Redis Persistence Overview

Redis is a critical component for handling high traffic and performance acceleration; any failure in the Redis service can cause business systems to crash. Understanding the core kernel design of Redis persistence is essential for building stable and reliable services.

Persistence Mechanisms

Redis provides three persistence methods: RDB (snapshot), AOF (append‑only file), and a hybrid of both. Persistence means writing the in‑memory data to durable storage such as SSD.

RDB (Redis Database)

RDB creates binary snapshots of the dataset. The format is compact binary data, and snapshots can be triggered manually, automatically, or passively. Redis offers three trigger modes and three fork‑based execution paths.

AOF (Append Only File)

AOF appends every write operation to a log in RESP (Redis Serialization Protocol) format, which is human‑readable text. Because the file grows monotonically, Redis provides an AOF rewrite mechanism that creates a new compact snapshot and then appends subsequent changes.

RDB Evolution and Baidu Cloud Optimizations

From version 2.0 to 7.0, Redis RDB has undergone ten major revisions. Key milestones include millisecond‑level expiration (2.5), CRC64 checksum (2.5), replication metadata (4.0), LRU/LFU caching (5.0), and functions with aof‑base (7.0).

2011 – Redis 2.5 adds millisecond expiration.

2012 – CRC64 checksum support.

2017 – Redis 4.0 adds replication metadata and aof‑preamble.

2018 – Redis 5.0 introduces LRU and LFU.

2022 – Redis 7.0 adds functions and aof‑base.

The traditional RDB process forks a child process to perform save, which then renames a temporary file to dump.rdb. Forking is a double‑edged sword: it enables non‑blocking snapshots but can cause blocking and memory bloat due to copy‑on‑write.

Baidu Cloud designed a “forkless” solution: the main process controls save progress while a worker thread performs the I/O, and the kernel uses COW to avoid data conflicts. It also introduces an ordered mixed snapshot format for RDB and AOF.

AOF Evolution and Baidu Cloud Practices

Key version milestones for AOF include the first rewrite mechanism (2013), parent‑child communication optimization (2015), RDB‑prefixed AOF (2017), and Multi‑Part AOF with extended annotations (2022, contributed by Baidu Cloud).

2013 – First AOF rewrite introduced.

2015 – Parent‑child communication optimization.

2017 – RDB‑prefixed AOF (mixed persistence).

2022 – Multi‑Part AOF with extended annotations.

Multi‑Part AOF splits the log into a base file (full snapshot) and incremental files. The main process still forks for rewrite, but only writes the base snapshot, while ongoing changes are appended to incremental files managed by a manifest.

Baidu Cloud further optimized AOF with asynchronous writes (delegating fsync to a worker thread), PSYNC‑AOF (partial synchronization based on AOF), and cross‑region active‑active replication using annotated AOF commands.

Future of Redis Persistence

Future persistence is expected to move from DRAM‑to‑SSD toward persistent memory, offering low latency and high performance. Baidu Cloud has already begun exploring this direction.