How Memcached Manages Memory: Slab Allocation, Eviction, and High‑Availability Design

1. Introduction to Memcached Memory Allocation

Understanding Memcached installation and basic commands is enough for simple development, but diagnosing production issues requires knowledge of its memory allocation mechanisms. Memcached uses a Slab Allocator to divide memory into fixed‑size chunks, eliminating fragmentation.

Item

An element to be stored, measured in bytes.

Chunk

The memory space that holds an item; think of it as a storage slot.

Slab Class

A group of chunks of the same size, e.g., 80 B, 96 B, etc.

Page

A memory block assigned to a slab class, defaulting to 1 MB. Each page is split into chunks according to the slab class size.

When Memcached starts, it can be configured with an initial slab size and a growth factor. For example, with an initial size of 80 B and a factor of 1.5, the slab class table is generated accordingly.

When a request for a 123 B item arrives, Memcached selects the smallest slab class that can accommodate it (e.g., 180 B). If that slab class has no allocated page, Memcached assigns one 1 MB page, which is then divided into 180 B chunks (approximately 5828 chunks). The item is stored in one of these chunks.

As memory fills, some slab classes may run out of pages while others still have free space. When all slabs are full and a new item arrives, Memcached triggers its eviction mechanism. It first checks the corresponding slab for expired items; if none exist, it evicts based on LRU within that slab only. Pages allocated to a slab are never reclaimed by other slabs until Memcached restarts, a phenomenon known as the "slab‑caching" or "calcium" problem.

2. High Concurrency & High Availability

2.1 Master‑Slave Dual‑Layer Structure

Data sharding distributes cache instances across multiple nodes, alleviating single‑node capacity limits. However, if a node fails, requests fall back to the backend database. Consistent hashing mitigates loss but can cause request drift and reduced cache hit rates in latency‑sensitive services.

To address single‑point failures, a master‑slave cache layer is introduced. The master handles writes; the slave serves as a backup. Reads first query the master; if the master returns empty, the slave is consulted. Updates use a CAS workflow on the master before propagating to the slave.

2.2 Horizontal Linear Scaling

Even with a dual‑layer structure, bandwidth saturation and request volume can exceed a single node's limits. Adding more replicas distributes load. An L1 cache layer placed beneath the master further improves scalability.

Write operations follow a master‑slave‑L1 sequence; failures trigger delete actions and subsequent cache‑through requests. L1 consists of independent cache groups; the application selects a group, hashes to a specific node, and performs multiget within that group. If L1 misses, the request falls back to master then slave, and successful reads are written back to L1.

When traffic reaches a threshold, additional L1 groups are added, achieving linear scaling. However, if the master’s hit rate is 99 %, the slave handles only 1 % of traffic, which may be insufficient if the master fails. To mitigate this, slaves are promoted as L1 resources, and the master is occasionally treated as an L1 group, balancing hot requests across both.

Source: http://blog.csdn.net/wongson/article/details/47418039