Why Is Redis So Fast? Architecture, Data Structures, Persistence, Replication and Cluster

Why Is Redis So Fast?

Many people only know that Redis is a K/V NoSQL in‑memory database with a single thread, but the real reasons for its speed lie in its memory‑first design, compact data structures, I/O multiplexing, and clever persistence and replication mechanisms.

How Fast Is It?

Official benchmarks show Redis can handle around 100,000 QPS. See the official benchmark program "How fast is Redis?" at https://redis.io/topics/benchmarks.

Memory‑Based Implementation

Redis stores all data in RAM, which is directly accessed by the CPU via the memory controller, giving it far lower latency than disk‑based databases.

Efficient Data Structures

Redis supports five data types: String, List, Hash, Set, SortedSet. Each type is backed by one or more specialized structures to maximize speed.

SDS – Simple Dynamic Strings

SDS stores the length of the string (O(1) length query), pre‑allocates extra space, and lazily frees unused space, reducing memory allocations.

ziplist – Compressed List

Used for List, Hash and SortedSet when the collection is small; stores small integers or short strings compactly.

quicklist – Hybrid List

Replaces ziplist + linked list in newer versions: a doubly linked list of ziplist segments, combining fast sequential access with low memory overhead.

skiplist – Skip List

Provides O(log N) ordered access for SortedSet by maintaining multiple forward pointers per node.

intset – Integer Set

When a Set contains only integers and is small, Redis stores it as a compact integer array.

Single‑Threaded Model

Redis’s single thread refers to the execution of key‑value commands and network I/O (since 6.0 network I/O can be multithreaded). Persistence, cluster sync and background deletion run in other threads.

Benefits of a single thread include no thread‑creation overhead, no context‑switch cost, no lock contention, and simpler code.

I/O Multiplexing Model

Redis uses epoll and a custom event loop to handle many client connections without blocking on any single socket.

Global Hash Dictionary

All keys are stored in a single hash table (array of buckets). Lookup is O(1) after computing the hash of the key.

The internal object is defined as:

<span style="color:#f92672;font-weight:bold;">typedef</span> <span style="color:#f92672;font-weight:bold;">struct</span> redisObject {<br/>    <span style="color:#75715e;">// type</span><br/>    <span style="color:#f92672;font-weight:bold;">unsigned</span> type:4;<br/>    <span style="color:#75715e;">// encoding</span><br/>    <span style="color:#f92672;font-weight:bold;">unsigned</span> encoding:4;<br/>    <span style="color:#75715e;">// pointer to the underlying data structure</span><br/>    <span style="color:#f92672;">void</span> *ptr;<br/>    <span style="color:#75715e;">// ...</span><br/>} robj;

Hash Collisions and Progressive Rehash

Redis uses chaining to resolve collisions. When a hash table becomes too full, a second table is allocated and entries are gradually moved (progressive rehash) to avoid blocking the server.

Persistence

Two main mechanisms:

RDB snapshots – a point‑in‑time dump of the entire dataset. Generated by forking a child process (COW) so the parent can continue serving requests.

AOF (Append‑Only File) – logs every write command. Supports three fsync policies: always, everysec, and no. A background rewrite ( bgrewriteaof) compacts the log.

Redis 4.0 introduced hybrid persistence, combining an RDB snapshot with the incremental AOF generated after the snapshot.

Master‑Slave Replication

Replication works in three phases: connection establishment, full sync (master creates an RDB and streams it), and incremental sync (new writes are sent via a replication buffer).

During network partitions, Redis uses an incremental sync buffer ( repl_backlog_buffer) and offsets ( master_repl_offset, slave_repl_offset) to resume replication without a full copy.

Sentinel

Sentinel monitors masters and slaves, detects failures, performs automatic failover, and notifies clients. It uses the special channel __sentinel__:hello on the master for inter‑sentinel communication and the INFO command to discover slaves.

Cluster

Redis Cluster shards data into 16,384 slots. Each key’s slot is computed as CRC16(key) % 16384. Slots are assigned to nodes, and nodes exchange slot maps via the Gossip protocol.

When a client contacts any node, the node returns the slot‑to‑node mapping so the client can route subsequent commands directly.

Redirection Mechanisms

MOVED – tells the client that the key resides on a different node (permanent). The client updates its slot cache.

ASK – used during slot migration; the client must send an ASKING command to the target node before the actual operation. The client’s slot cache is not updated.

Conclusion

The combination of in‑memory storage, compact data structures, a simple single‑threaded command model with epoll, efficient hash tables, progressive rehashing, flexible persistence, robust replication, Sentinel monitoring, and a sharded Cluster gives Redis its remarkable speed and scalability.