Understanding Redis Persistence: RDB, AOF, Hybrid Persistence and Redis 7 Multi‑Part AOF

Why persistence matters

At 3 a.m. a data‑center lost power. After services were restarted, all session, counter and leaderboard data stored in Redis vanished, and a flood of requests hit MySQL, overwhelming it again. This demonstrates the fundamental weakness of in‑memory databases: when the process stops, data can be lost.

Redis persistence overview

Redis provides two independent persistence mechanisms that can be used alone or together (hybrid persistence):

AOF (Append‑Only File) : every write command is appended to a log file and replayed on restart.

RDB (Redis DataBase) : a binary snapshot of the entire memory state is taken at a point in time.

RDB snapshots

RDB creates a compact binary file (default dump.rdb) that can be loaded directly, making recovery fast and the file small. The typical BGSAVE workflow is:

Trigger a snapshot manually with SAVE or BGSAVE, or let the save configuration fire automatically.

Redis forks a child process; the child sees a copy‑on‑write (COW) view of the memory.

The child writes the snapshot to a temporary file.

The parent continues serving clients without blocking.

After the child finishes, the temporary file atomically replaces the old dump.rdb.

Because only the state at the moment of the fork is saved, any writes that occur between two snapshots are lost. RDB is therefore fast to load but offers lower data safety.

RDB core configuration

# RDB file name
 dbfilename dump.rdb
# Directory for snapshots (use a dedicated data disk)
 dir /data/redis/6379
# Automatic snapshot triggers (default)
 save 900 1   # 1 change in 15 min
 save 300 10  # 10 changes in 5 min
 save 60 10000 # 10 000 changes in 1 min
# Enable compression (saves space, adds a small CPU cost)
 rdbcompression yes
# Write checksum for integrity checking
 rdbchecksum yes
# Stop writes if a snapshot fails (helps detect problems early)
 stop-writes-on-bgsave-error yes

Key points: save controls automatic snapshot frequency; more writes trigger snapshots more often. dir decides where RDB files are stored; production should use a reliable disk. stop-writes-on-bgsave-error yes rejects writes when a snapshot fails, preventing silent persistence loss. rdbcompression yes saves space at a modest CPU cost. rdbchecksum yes improves file reliability.

Validating RDB

# 1. Write a test key
127.0.0.1:6379> SET rdb:test "hello-rdb"
# 2. Verify configuration
127.0.0.1:6379> CONFIG GET dir
127.0.0.1:6379> CONFIG GET dbfilename
# 3. Trigger a background snapshot
127.0.0.1:6379> BGSAVE
# 4. Check persistence status
127.0.0.1:6379> INFO persistence

Successful indicators include rdb_last_bgsave_status:ok, rdb_bgsave_in_progress:0, and the presence of dump.rdb in the configured directory.

AOF command logging

AOF records every write command in the exact RESP format. On restart, Redis replays the log to reconstruct the dataset, usually yielding a more recent state than RDB.

Client sends a write command, e.g. SET user:1 tom.

Redis updates the in‑memory data.

The command is appended to the AOF buffer.

The buffer is flushed to the AOF file according to the appendfsync policy.

On restart, the file is read and each command is executed in order.

AOF core configuration

# Enable AOF
 appendonly yes
# Base file name (Redis 7+ will split it into multiple files)
 appendfilename "appendonly.aof"
# Directory for AOF files (default "appendonlydir")
 appenddirname "appendonlydir"
# Flush policy (recommended)
 appendfsync everysec   # fsync at most once per second
# Skip fsync during rewrite to reduce latency (optional)
 no-appendfsync-on-rewrite yes
# Automatic rewrite when size grows 100% and exceeds 64 MB
 auto-aof-rewrite-percentage 100
 auto-aof-rewrite-min-size 64mb
# Use RDB format as the base part of a rewrite (hybrid persistence)
 aof-use-rdb-preamble yes
# Allow truncating a partially written file on start‑up
 aof-load-truncated yes

AOF rewrite process

Because the log grows indefinitely, Redis periodically rewrites it:

Fork a child process; the child scans the current memory and writes a minimal set of commands that can recreate the same state.

During the rewrite the parent continues writing new commands to the existing AOF and also to a temporary rewrite buffer.

When the child finishes, the parent appends the buffered commands to the new AOF and atomically replaces the old file.

Redis 7 replaces the rewrite buffer with a multi‑part layout, eliminating the extra memory and I/O overhead.

Redis 7 Multi‑Part AOF

Starting with Redis 7, AOF is stored as three files inside appendonlydir:

base : a full snapshot in RDB format (fast to load).

incr : the incremental command log written after the base.

manifest : a plain‑text index that lists the current base and incr files.

During a rewrite a new incr file is created immediately, so new writes never need a separate rewrite buffer. The child writes a new base file, the manifest is updated to point to the new base+incr pair, and the old files are deleted.

Classic vs. Multi‑Part AOF comparison

Files : classic uses a single appendonly.aof; multi‑part uses base, incr, manifest.

Writes during rewrite : classic requires dual‑write to the current AOF and a rewrite buffer; multi‑part writes directly to the new incr file.

Rewrite buffer : required in classic (extra memory & I/O); not needed in multi‑part.

Rewrite completion : classic appends buffered data to the new file; multi‑part updates the manifest and deletes old files.

Replication and persistence

Full‑sync replication reuses the RDB mechanism: the master creates a snapshot and streams it to the replica (optionally disk‑less via repl-diskless-sync yes). Therefore even a master that only needs replication must be able to generate RDB files.

Master : should enable persistence (AOF for safety, RDB for fast restarts) to avoid an “empty master” after a crash.

Replica : at least RDB is recommended so the replica can restart quickly without pulling a full sync again.

Common pitfalls

Disabling persistence on the master and letting it auto‑restart leads to total data loss across the replica set.

Too small repl-backlog-size forces frequent full syncs, increasing load.

Large instance memory causes long fork pauses during snapshots or rewrites.

AOF files are not a substitute for proper backups; they also record erroneous commands.

Selecting the right strategy

Choose a persistence scheme based on how much data loss your application can tolerate:

Pure cache, data can be rebuilt : RDB only or even none (re‑populate from DB).

Critical session / counter data : AOF (everysec) + hybrid (RDB preamble).

Need fast recovery *and* high safety : Hybrid persistence (AOF + RDB base).

Cold backup / disaster‑recovery : Enable RDB (small, easy to copy).

Ultra‑low latency writes : Consider disabling AOF or using no fsync, but accept higher risk.

Practical validation steps

# Verify persistence status
127.0.0.1:6379> INFO persistence
# Write a test key, trigger a snapshot and an AOF rewrite
127.0.0.1:6379> SET persist:test "alive"
127.0.0.1:6379> BGSAVE
127.0.0.1:6379> BGREWRITEAOF
# Restart Redis and confirm the key is still present
127.0.0.1:6379> GET persist:test   # should return "alive"

Key takeaways

AOF provides the highest durability (loss ≤ 1 s with everysec) but larger files and slower restarts.

RDB offers tiny snapshots and rapid recovery but can lose up to the interval between snapshots.

Hybrid persistence (AOF with RDB preamble) gives the best of both worlds.

Redis 7’s Multi‑Part AOF removes the rewrite buffer, reducing memory pressure and I/O during compaction.

Never rely on Redis as the sole source of truth for business‑critical data; always have a durable backing store.