How Redis Prevents Data Loss During Crashes

Preface

If you are asked which business scenarios are suitable for Redis, you will likely answer that it is used as a cache because it stores data in memory and reads directly from memory, offering very fast response times. However, an unavoidable issue is that if the server crashes, all data in memory is lost .

Redis provides two major persistence mechanisms: AOF ( Append Only File ) logging and RDB ( Redis DataBase ) snapshots .

AOF

How the log is implemented

Unlike the write‑ahead log (WAL) used by many databases, AOF is a write‑behind log: Redis first executes the command, writes data to memory, and then records the command.

AOF records every command received by Redis as plain text. For example, after receiving the command set testkey testvalue, the log contains:

Advantages and risks of write‑behind logging

Advantages :

Redis does not perform syntax checking before logging, so only successfully executed commands are recorded, avoiding erroneous command entries.

The log is written after the command executes, so it does not block the current write operation .

Risks :

If a command finishes execution but the log has not been written before a crash, that command and its data may be lost. When Redis is used as a cache, the data can be re‑loaded from the backend database; when Redis is the primary database, the loss cannot be recovered from the log.

Although AOF avoids blocking the current command, it can cause blocking for the next operation when the log is flushed to disk, especially if the disk write pressure is high.

Both risks are related to the timing of writing AOF entries to disk. Controlling when the log is flushed can mitigate them.

Log write‑back strategies

The AOF mechanism offers three appendfsync options:

Always : after each write command, the log is synchronously flushed to disk.

Everysec : commands are first written to an in‑memory buffer; the buffer is flushed to disk every second.

No : commands are written to the in‑memory buffer and the operating system decides when to flush.

These strategies trade off between performance and data safety:

Choose No for highest performance.

Choose Always for strongest durability.

Choose Everysec when a small amount of data loss is acceptable but performance impact must be limited.

Log rewriting

Purpose of rewriting

Because AOF records every command, the file grows continuously, leading to three problems:

File‑system size limits may be reached.

Appending to a very large file becomes inefficient.

During crash recovery, replaying a huge AOF file can be extremely slow.

Rewriting process

Redis forks a background subprocess bgrewriteaof to perform rewriting, avoiding main‑thread blockage. The process can be described as “one copy, two logs”:

First log : while the main thread continues handling new commands, those commands are written to the in‑memory buffer of the original AOF.

Second log : the background subprocess writes a new compacted AOF file that contains a single command for each key‑value pair, effectively converting many incremental commands into one.

Because rewriting is done in a child process, the main thread is not blocked, and the new AOF file can replace the old one after rewriting completes.

RDB (Snapshot)

RDB creates a point‑in‑time snapshot of the entire dataset and writes it to disk as an .rdb file. This snapshot can be loaded directly into memory after a crash, providing fast recovery.

Redis offers two commands to generate snapshots:

save : executed in the main thread and blocks other operations.

bgsave : forks a child process to write the snapshot, avoiding main‑thread blockage (the default configuration).

The snapshot process uses the operating system's copy‑on‑write ( Copy-On-Write) mechanism: the child process shares the parent’s memory pages, and only modified pages are duplicated, allowing the main thread to continue serving reads and writes while the snapshot is being written.

Hybrid AOF/RDB

Frequent full snapshots can cause two issues:

Heavy disk I/O from repeatedly writing large snapshots, leading to competition for limited bandwidth.

The fork operation itself blocks the main thread, and the blocking time grows with the size of the dataset.

Redis 4.0 introduced a hybrid approach: periodic RDB snapshots are taken, and between two snapshots all write commands are recorded in an AOF log. This reduces the frequency of costly forks and limits AOF file growth because only the commands between snapshots are logged.

Summary and Recommendations

When data loss is unacceptable, combine memory snapshots with AOF for strong durability.

If minute‑level data loss is tolerable, using only RDB is sufficient.

If using only AOF, prefer the Everysec configuration to balance reliability and performance.