Unlocking Redis Memory: How Its Internal Model Impacts Performance

Preface

Redis is one of the hottest in‑memory databases; by reading and writing data directly in memory it dramatically speeds up operations, making it indispensable for high‑concurrency websites. Redis exposes five object types (string, hash, list, set, sorted set). Understanding their usage and the underlying memory model helps estimate memory consumption, optimize usage, and diagnose issues.

1. Redis Memory Statistics

After connecting with redis-cli, the INFO MEMORY command displays memory‑related information. Important fields include:

used_memory : total memory allocated by Redis's allocator (bytes), including virtual memory.

used_memory_rss : memory the Redis process occupies in the OS (bytes), matching top or ps output; it also includes process overhead and fragmentation.

mem_fragmentation_ratio : ratio of used_memory_rss to used_memory. Values > 1 indicate fragmentation; values < 1 suggest swapping.

mem_allocator : memory allocator used (default jemalloc).

2. Redis Memory Partition

Redis memory can be divided into four parts:

Data : the key‑value pairs stored in used_memory.

Process memory : code, constants, and other runtime data (a few megabytes, not counted by jemalloc).

Buffer memory : client buffers, replication backlog, AOF buffer, all allocated by jemalloc and counted in used_memory.

Memory fragmentation : unused gaps caused by allocation/deallocation patterns; not counted in used_memory.

3. Redis Data‑Storage Details

3.1 Overview

Key components include dictEntry, Key (stored as SDS), redisObject, and the memory allocator ( jemalloc). When executing SET hello world, the key and value are both wrapped in SDS and redisObject.

3.2 jemalloc

Redis is compiled with a memory allocator that can be libc, jemalloc (default), or tcmalloc. jemalloc reduces fragmentation by categorizing allocations into small, large, and huge size classes and selecting the best‑fit block.

3.3 redisObject

typedef struct redisObject {
    unsigned type:4;          // object type (string, list, hash, set, zset)
    unsigned encoding:4;    // internal encoding
    unsigned lru:REDIS_LRU_BITS; // last access time
    int refcount;            // reference count
    void *ptr;               // pointer to actual data
} robj;

Fields:

type : identifies the object type.

encoding : specifies the internal representation (e.g., int, embstr, raw for strings).

lru : last access timestamp, used for idle‑time tracking and LRU eviction.

refcount : number of references; when it reaches zero the object is freed.

ptr : points to the actual data (e.g., an SDS buffer).

3.4 Simple Dynamic String (SDS)

typedef struct sdshdr {
    int len;   // used length
    int free;  // unused space
    char buf[]; // character buffer (null‑terminated)
} sdshdr;

SDS stores length and free space, offering O(1) length queries, automatic buffer resizing, and safe binary data handling—advantages over traditional C strings.

4. Redis Object Types and Internal Encodings

Each of Redis's five object types has at least two internal encodings, allowing the server to choose the most memory‑efficient representation based on size and usage patterns.

4.1 String

int

: 8‑byte integer when the value fits. embstr: ≤ 39 bytes, stored as a single contiguous allocation (object + SDS). raw: > 39 bytes, separate allocations for object and SDS.

4.2 List

ziplist

: compact sequential memory for ≤ 512 elements and each element ≤ 64 bytes. linkedlist: doubly‑linked list for larger lists.

4.3 Hash

ziplist

: when field count < 512 and each field/value ≤ 64 bytes. hashtable: otherwise.

4.4 Set

intset

: when all members are integers and count < 512. hashtable: otherwise (values are NULL).

4.5 Sorted Set (Zset)

ziplist

: when member count < 128 and each member ≤ 64 bytes. skiplist: otherwise.

5. Practical Applications

5.1 Estimating Memory Usage

For 90 000 key‑value pairs where each key and value is 7 bytes (non‑integer), the encoding is embstr. Each dictEntry occupies 32 bytes (jemalloc round‑up), each key/value SDS occupies 16 bytes, and each redisObject occupies 16 bytes, totaling 80 bytes per entry. The hash table (bucket) size is the next power of two (131 072) × 8 bytes = 1 048 576 bytes. Estimated memory: 90 000 × 80 + 1 048 576 ≈ 8 248 576 bytes, matching the measured value (8 247 552 bytes).

5.2 Optimizing Memory Consumption

Leverage jemalloc size classes: reducing key/value length from 8 bytes to 7 bytes halves the allocation for each SDS.

Store numeric data as integers to use the 8‑byte int encoding.

Enable shared objects (default 0‑9999) or increase REDIS_SHARED_INTEGERS to reuse common integer objects.

Avoid over‑engineering; memory savings must outweigh added complexity.

5.3 Monitoring Fragmentation Ratio

A healthy mem_fragmentation_ratio is around 1.03 for jemalloc. Ratios > 1 indicate fragmentation; a restart can defragment memory. Ratios < 1 imply swapping; adding physical memory or reducing data size is required.

Code Example (Java with Jedis)

public class RedisTest {
    public static Jedis jedis = new Jedis("localhost", 6379);

    public static void main(String[] args) throws Exception {
        Long m1 = Long.valueOf(getMemory());
        insertData();
        Long m2 = Long.valueOf(getMemory());
        System.out.println(m2 - m1);
    }

    public static void insertData() {
        for (int i = 10000; i < 100000; i++) {
            jedis.set("aa" + i, "aa" + i); // key and value length = 7 bytes, not integers
        }
    }

    public static String getMemory() {
        String memoryAllLine = jedis.info("memory");
        String usedMemoryLine = memoryAllLine.split("
")[1];
        String memory = usedMemoryLine.substring(usedMemoryLine.indexOf(':') + 1);
        return memory;
    }
}