Why Redis Is More Than a Cache: Architecture, Persistence, and Scaling Explained

What is Redis

Redis (Remote DIctionary Service) is an open‑source, in‑memory key‑value store that provides a rich set of data structures (strings, hashes, lists, sets, sorted sets, streams, etc.). It is commonly used as a fast cache in front of a persistent database (e.g., MySQL, PostgreSQL) but can also serve as a primary datastore when combined with high‑availability configurations.

Redis deployment models

Single Redis instance – simplest setup, suitable for small workloads; a single point of failure.

Master‑slave replication – asynchronous replication from a writable master to one or more read‑only slaves.

Redis Sentinel – distributed processes that monitor masters and slaves, perform automatic failover, and provide service discovery.

Redis Cluster – horizontal scaling by sharding data across multiple nodes using hash slots.

1. Single instance

A single instance runs in a single process, processes commands directly in memory, and can optionally enable persistence. Persistence is achieved via a background fork() that writes either an RDB snapshot or an AOF file without blocking the main thread.

2. Master‑slave replication (high availability)

Each master maintains a replication ID and a monotonically increasing offset . Slaves track these values to decide whether a partial sync (few offsets behind) or a full sync (offset mismatch) is required. During a full sync the master creates a new RDB snapshot and streams it to the slave. The master continues to buffer commands that arrive after the snapshot point so they can be replayed on the slave once the snapshot transfer completes.

3. Redis Sentinel

Sentinel nodes continuously monitor the health of masters and slaves. Their responsibilities are:

Monitoring – ping masters and slaves to ensure they are responsive.

Notification – alert administrators when a node fails.

Failover management – when a quorum of sentinel processes agrees that a master is down, they promote one of its replicas to master.

Configuration discovery – act as a service‑discovery layer so clients can obtain the current master address.

Typical deployments run at least three sentinel processes and configure a quorum of two to avoid split‑brain scenarios. An odd number of master nodes with at least two replicas per master is recommended.

4. Redis Cluster

Cluster shards data across multiple nodes using 16,384 hash slots . A key is hashed, the hash value is taken modulo 16,384, and the resulting slot determines the owning node. Adding or removing a node triggers resharding , which moves whole hash‑slot ranges (not individual keys) between nodes.

Example slot allocation:

M1 – slots 0‑8191

M2 – slots 8192‑16383

When a third master M3 is added, the slots can be redistributed, e.g.:

M1 – slots 0‑5460

M2 – slots 5461‑10922

M3 – slots 10923‑16383

Only the slot ranges move; keys within a slot stay where they are, which makes resharding fast and minimizes downtime.

5. Gossip protocol

Cluster nodes exchange health information via a gossip protocol. If a majority of masters agree that a master is unreachable, they can promote one of its replicas. Proper quorum sizing (odd number of masters, at least two replicas) prevents split‑brain conditions.

Persistence models

No persistence – fastest operation; all data is lost on restart.

RDB (Redis Database) – periodic point‑in‑time snapshots saved to disk. Configurable via save directives (e.g., save 900 1 for a snapshot every 900 seconds if at least one key changed). Fast restart but may lose up to the interval between snapshots.

AOF (Append‑Only File) – logs every write command. The appendfsync policy can be set to always, everysec, or no. Provides higher durability at the cost of larger disk usage and slower writes.

RDB + AOF – both mechanisms enabled; on restart Redis prefers AOF for reconstruction because it contains the most recent operations.

Forking for persistence

Redis uses the operating system’s fork() to create a child process that writes the snapshot or AOF file. Thanks to copy‑on‑write, the parent and child share memory pages until a write occurs, allowing a point‑in‑time snapshot without duplicating the entire memory footprint. This approach keeps the main event loop responsive while the child persists data.