Unlocking Redis Memory: Deep Dive into Its Internal Model and Optimization

Redis Memory Model Overview

Redis is a popular in‑memory database that dramatically speeds up read/write operations; understanding its memory model is essential for estimating usage, optimizing consumption, and troubleshooting issues.

Redis Memory Statistics

After connecting with redis-cli, the INFO MEMORY command displays memory‑related fields. Key fields include:

used_memory : total memory allocated by the allocator (bytes), including virtual memory.

used_memory_rss : memory the Redis process occupies in the OS (bytes), matching values from top or ps.

mem_fragmentation_ratio : ratio of used_memory_rss to used_memory, indicating fragmentation.

mem_allocator : allocator used (libc, jemalloc, or tcmalloc); default is jemalloc.

Redis Memory Partition

Memory consumption can be divided into four parts:

Data

All key‑value pairs are stored as data and counted in used_memory. Redis supports five object types (string, hash, list, set, sorted set), each with internal encodings.

Process Memory

The Redis server binary, constant pool, and other code occupy a few megabytes; this memory is not allocated by jemalloc and is not included in used_memory.

Buffer Memory

Client buffers, replication backlog, and AOF buffers are allocated by jemalloc and therefore appear in used_memory.

Memory Fragmentation

Fragmentation arises from allocation/deallocation patterns and the allocator design. High fragmentation inflates used_memory_rss relative to used_memory.

Details of Redis Data Storage

Overview

Key concepts include the memory allocator (jemalloc), Simple Dynamic Strings (SDS), the five object types, and the redisObject wrapper that stores type, encoding, LRU, refcount, and a pointer to the actual data.

jemalloc

Redis defaults to jemalloc, which reduces fragmentation by grouping allocations into size classes (small, large, huge). For example, a 130‑byte object is placed into a 160‑byte slot.

redisObject

The redisObject structure (16 bytes) holds:

type : 4 bits indicating one of the five object types.

encoding : 4 bits describing the internal representation.

lru : timestamp for LRU eviction.

refcount : reference count for shared objects.

ptr : pointer to the actual data (e.g., an SDS string).

SDS (Simple Dynamic String)

SDS extends C strings with len and free fields, enabling O(1) length queries, automatic reallocation, and safe binary data handling.

Redis Object Types and Internal Encodings

String

Three encodings: int (8 bytes), embstr (≤39 bytes, stored contiguously), and raw (>39 bytes). Embstr is read‑only; modifications trigger conversion to raw.

List

Internal encodings are ziplist (compact) and linkedlist (standard). Ziplist is used when the list has < 512 elements and each element < 64 bytes; otherwise Redis switches to linkedlist.

Hash

Inner hashes use ziplist or hashtable depending on element count (<512) and key/value length (<64 bytes). The outer hash (the key‑value store) always uses a hashtable.

Set

Encodings are intset (for small integer sets) or hashtable. Intset is chosen when the set has <512 elements and all members are integers.

Sorted Set

Encodings are ziplist (for ≤128 elements and member length < 64 bytes) or skiplist (default). Skiplist provides O(log N) average lookup.

Practical Applications

Estimating Memory Usage

By analyzing object sizes (dictEntry 24 bytes, SDS len+9, redisObject 16 bytes) and bucket allocation (next power‑of‑two), one can compute approximate memory consumption for a given number of keys.

Optimizing Memory Consumption

Strategies include using shorter keys/values to fit smaller jemalloc classes, preferring integer types over strings, and increasing the shared‑integer pool (default 0‑9999) to reduce duplicate objects.

Monitoring Fragmentation Ratio

A healthy mem_fragmentation_ratio is around 1.03. Values >1 indicate fragmentation; values <1 suggest swapping to virtual memory, prompting a restart or memory upgrade.