Inside Redis AOF Rewrite: How Forked Processes Manage Data Persistence

The article offers a comprehensive walkthrough of Redis's Append‑Only File (AOF) rewrite mechanism, explaining how the system triggers a rewrite, forks a child process, coordinates via three pipes, manages multiple in‑memory buffers, and finally replaces the old AOF with the newly rewritten file.

Triggering AOF Rewrite

Redis can start an AOF rewrite in three ways: (1) automatically when the configured time interval and change‑count thresholds are exceeded (checked inside serverCron), (2) manually via the BGREWRITEAOF command issued by a client, and (3) once at startup when AOF is enabled, after the initial RDB sync in a master‑slave setup.

Parent and Child Process Interaction

When a rewrite is needed, Redis forks a child process. The parent continues to serve client requests while the child rewrites the AOF. Fork uses copy‑on‑write, so the child initially shares the parent’s memory pages; any write causes a private copy, ensuring the child sees a consistent snapshot of the data.

Communication Channels

Three unidirectional pipes are created between the parent and child:

Pipe 1 carries incremental changes (diffs) generated by the parent while the child is rewriting.

Pipe 2 is used by the child to signal the parent to stop sending diffs (the child writes a “!” when it reaches about 80 % of the rewrite).

Pipe 3 is used by the parent to acknowledge the stop request and to notify the child that it will no longer receive data.

Key Data Structures

aof_buf

– the parent’s primary buffer for AOF writes. aof_rewrite_buf_blocks – a list of 10 MiB blocks that temporarily store diffs while the rewrite is in progress. aof_child_diff – the child’s structure that accumulates changes read from Pipe 1.

Child Process Rewrite Steps

Create a temporary file named temp‑rewriteaof‑{pid} and initialise its file handle.

Determine whether Redis is operating in pure AOF mode or mixed RDB‑AOF mode.

Iterate over each database, converting commands to RESP format and writing them sequentially to the new file.

Parent Process During Rewrite

While the child rewrites, the parent continues to accept client commands. Each command is written to aof_buf (the active AOF) and also appended to aof_rewrite_buf_blocks so that changes are not lost. When the child signals that it has processed 80 % of the data, it writes “!” to Pipe 2; the parent then sets aof_stop_sending_diff to stop feeding Pipe 1 and writes its own “!” to Pipe 3.

Finalization

After the child finishes rewriting the old data, it reads any remaining diffs from Pipe 1, flushes them to disk, and exits. The parent, noticing the child’s termination in serverCron, calls backgroundRewriteDoneHandler which:

Writes any leftover blocks from aof_rewrite_buf_blocks into the new AOF file ( temp‑rewriteaof‑bg‑{pid}.aof).

Renames this temporary file to become the active AOF, retiring the old file.

Performance Considerations

Forking copies the page table, which can block the process; heavy write traffic may increase latency because the parent must duplicate pages on write. Redis 5.0 uses two memory buffers and three pipes, effectively doubling CPU and memory overhead during a rewrite. Redis 7.0 introduces Multi‑Part AOF to reduce this redundancy.

Tip: Because the forked child shares memory with the parent, it must know where the parent’s original data resides. The page‑table copy is a blocking operation that can stall the fork. During high traffic, AOF rewrite may increase response time.