Raft Log Replication: Format, Process, and Consistency Guarantees

Raft Log Format

In the Raft algorithm, the replicated state machine's data is stored in a log. Unlike typical application logs, each log entry (LogEntry) is appended in order with a fixed format that includes a command, the term of the leader that created it, and the entry's index.

As shown, each log entry consists of a command (the client request to be stored), an index (a continuously increasing integer), and a term number (the leader's term when the entry was created).

Command: The operation sent by the client, essentially the data that needs to be stored.

Index: The position of the entry in the log; it is a monotonically increasing integer.

Term: The term of the leader that created this entry.

Log Replication Process

The replication process works as follows:

The leader receives a client request, creates a new log entry, and appends it to its local log. It then sends an AppendEntries RPC to followers, copying the new entry to their logs. Once a majority of followers acknowledge success, the leader applies the entry to the state machine and returns a success response to the client.

This flow shows that after the leader receives acknowledgments from a majority of followers and applies the entry, it does not need to send a response back to the followers; the success is reported directly to the client. The leader also includes the newly committed entry in the next AppendEntries RPC.

If no failures occur, this is the normal replication path. However, node crashes can happen, and Raft must handle them.

How to Ensure Log Consistency?

Under normal conditions, successful AppendEntries responses keep leader and follower logs consistent. When a leader crashes, inconsistencies may arise, especially if multiple leaders fail in succession.

Note: The example is taken from the Raft paper.

When a leader is elected, followers may be in one of the following states (a‑f):

a‑b: Followers are behind the current leader.

c‑d: Followers have some entries that are not yet committed.

e‑f: A more complex mix of the above situations.

The article reconstructs a scenario where leaders e, f, a, c, and d are elected in successive terms, each experiencing crashes and partial log replication, leading to the state shown in the diagram.

Raft resolves consistency using two crucial properties:

If two entries in different logs have the same index and term, they store the same command.

If two entries have the same index and term, then all preceding entries are also identical.

The first property holds because log entries are immutable; matching index and term imply identical commands. The second property enables the leader to force followers to overwrite divergent entries: during AppendEntries, the leader includes the previous log index and term. If a follower cannot find a matching entry, it rejects the request, and the leader decrements the index until it finds a match, then overwrites all subsequent entries.

Thus, log appending consists of two steps:

The leader finds the follower’s highest matching log entry, ensuring all prior entries are identical.

The leader forces the follower to overwrite any later inconsistent entries, achieving consistency.

An illustrative example shows how forced overwriting works:

Assume a leader and a follower with the following log states:

The follower was a leader in term 3, crashed during log appending, and after restart became a follower. A new leader then appends entries; the follower’s last entry from term 3 is overwritten.

AppendEntries RPC request parameters are:

Parameter

Description

term

Leader’s current term

leaderId

Leader’s ID (used by followers to redirect clients)

prevLogIndex

Index of the log entry immediately preceding the new entries

prevLogTerm

Term of the entry at prevLogIndex

entries[]

Log entries to store (empty when used as a heartbeat; multiple entries may be sent for efficiency)

leaderCommit

Leader’s highest known committed entry index

The leader’s forced overwrite process proceeds as follows:

The leader sends AppendEntries with the latest entries and prevLogIndex=7, prevLogTerm=3.

The follower detects a term mismatch for its latest entry and rejects the request.

The leader decrements the index to prevLogIndex=6, prevLogTerm=3 and retries.

The follower finds a matching entry at index 6, term 3, and accepts the request.

The leader then overwrites all entries after index 6 on the follower, achieving consistency.