Memcached vs Redis: Architecture, Memory Management & Persistence

Overview

Memcached and Redis are the two most widely used in‑memory cache servers. Both store data as key‑value pairs in RAM to accelerate read‑heavy workloads, reducing database queries and improving response time.

Service Model

Each server runs as an independent daemon process and exposes a network service. Clients connect via TCP (default), with optional UDP support and Unix‑domain sockets for same‑host communication. The server accepts connections, parses commands, and returns results over the network.

Event Model

Both servers use epoll (or kqueue on BSD) for I/O multiplexing. Redis implements a single‑threaded reactor loop; background tasks (e.g., persistence) run in auxiliary threads. Memcached uses a master‑worker model: a master thread accepts connections and distributes them to multiple worker threads, each with its own epoll loop.

Redis maps file descriptors to client structures using a red‑black tree (O(log n)). When the maximum number of connections is known, it stores client pointers in an array indexed by fd, achieving O(1) lookup. Memcached workers communicate with the master via pipes and maintain per‑thread ready‑connection queues.

Memory Allocation

Memcached implements a slab allocator. A large memory pool is pre‑allocated and divided into classes of fixed‑size slabs; each slab contains multiple trunk s. Items are allocated from the slab class matching the item size and returned to the slab’s free list on deletion. This reduces fragmentation but can waste space when an item’s size does not exactly match the slab size.

Redis relies on the system allocator ( malloc / free) and can optionally replace glibc’s malloc with Google’s tcmalloc for better performance. Objects are allocated as redisObject structures, so allocation is straightforward but involves more system calls compared with a slab pool.

Database Implementation

Memcached stores only simple key‑value pairs. Each pair is represented by an item struct containing metadata, the key, and the value. Items are placed in a hash table that uses separate chaining. The hash table expands dynamically: when the load factor exceeds 1.5, a larger table is allocated and entries are gradually moved by a background thread. Expiration is lazy; a key is checked for expiry only when accessed. To avoid lost updates, Memcached uses a Compare‑And‑Swap (CAS) protocol: each item carries a 64‑bit version counter that must match the client‑provided value before an update succeeds.

Redis supports multiple data structures (string, list, set, sorted set, hash). Each stored value is wrapped in a redisObject that records type, encoding, reference count, and LRU timestamp. Encodings vary per type (e.g., raw strings, integer‑encoded strings, embedded strings, ziplist, skiplist). Redis uses a dictionary ( dict) with two hash tables (primary and old) to enable incremental rehashing without stopping the event loop. Sorted sets use a skiplist (O(log n) operations) optionally backed by a ziplist for small collections.

Redis also provides multiple logical databases (default 16). Keys are isolated per database, and an additional expireDict stores expiration timestamps only for keys that have a TTL, keeping memory usage low.

Persistence

RDB (Redis Database) snapshots the entire dataset to a binary file when the user issues SAVE or BGSAVE. The file begins with a magic string and version, followed by each database’s key‑value pairs, prefixed by a SELECTDB opcode and optional EXPIRETIME_MS fields. BGSAVE forks a child process that writes the snapshot while the parent continues serving clients. The parent tracks a “dirty” counter (writes since the last snapshot) to decide when background saves should be triggered.

AOF (Append‑Only File) logs every write command as it arrives. Commands are buffered in aof_buf and flushed to the OS buffer each event‑loop iteration. A background rewrite ( AOF_REWRITE) creates a compact representation of the current dataset by generating the minimal set of commands needed to reconstruct it. The rewrite runs in a child process; the parent continues serving clients and records subsequent write commands in a linked list ( aof_rewrite_buf_blocks) that are appended to the new AOF after the child exits.

Loading an AOF replays the logged commands using a fake client that reads from the file and executes each command directly.

Transactions

Redis provides a simple transaction model using MULTI / EXEC. Commands issued after MULTI are queued in the client’s multiState and executed atomically when EXEC is received. There is no rollback; if a command fails, the error is recorded but execution continues. The WATCH command offers optimistic concurrency control: keys watched before MULTI are monitored, and if any are modified by another client before EXEC, the transaction aborts with a “dirty” error.

Publish/Subscribe

Redis implements pub/sub using a pubsub_channels dictionary that maps channel names to a list of subscribed client structures. Pattern subscriptions are stored in a pubsub_patterns list. When a message is published, Redis iterates over the channel’s client list and over matching pattern subscriptions, delivering the message to each client.

Conclusion

Redis offers a richer feature set—multiple data structures, configurable persistence, transactions, and pub/sub—at the cost of greater implementation complexity. Memcached focuses on a simple, high‑performance key‑value cache with a slab‑based memory allocator and a multi‑threaded design. From a source‑code study perspective, Redis provides more material to explore, while Memcached remains an excellent example of a lightweight, single‑purpose cache.