Unlocking Redis Memory: Deep Dive into Its Internal Model and Optimization

1. Introduction

Redis is one of the most popular in‑memory databases, offering high read/write speed by storing data directly in memory. It supports five object types (string, hash, list, set, sorted set), which give it an advantage over alternatives like Memcached.

Understanding Redis's memory model helps in estimating memory consumption, optimizing usage, and diagnosing issues such as blocking or high memory fragmentation.

2. Redis Memory Statistics

Use the INFO MEMORY command via redis-cli to view memory metrics. Key fields include:

used_memory : total memory allocated by Redis (including virtual memory).

used_memory_rss : memory occupied by the Redis process as seen by the OS.

mem_fragmentation_ratio : ratio of used_memory_rss to used_memory, indicating fragmentation.

mem_allocator : memory allocator used (libc, jemalloc, or tcmalloc; default is jemalloc).

3. Redis Memory Partition

Memory consumption can be divided into:

1) Data

Stored in used_memory. Redis objects (string, hash, list, set, sorted set) are wrapped in internal structures such as redisObject and SDS.

2) Process Overhead

The Redis server process itself uses a few megabytes for code, constants, etc., not counted in used_memory. Child processes (e.g., for AOF or RDB rewriting) also consume memory but are excluded from the main metrics.

3) Buffers

Client buffers, replication backlog, and AOF buffers are allocated by jemalloc and included in used_memory.

4) Memory Fragmentation

Fragmentation occurs during allocation and deallocation. Jemalloc reduces fragmentation, but high mem_fragmentation_ratio still indicates wasted space.

4. Details of Redis Data Storage

4.1 Overview

Key concepts include the memory allocator (jemalloc), Simple Dynamic Strings (SDS), the five object types, their internal encodings, and redisObject. The diagram below shows the data model for SET hello world:

4.2 jemalloc

Jemalloc is the default allocator for Redis, dividing memory into small, large, and huge classes and further into size classes. For example, a 130‑byte object is placed in a 160‑byte slot.

4.3 redisObject

All Redis objects are represented by redisObject, which contains fields such as type , encoding , lru , refcount , and ptr . The structure occupies 16 bytes.

type

Indicates the object type (string, list, hash, set, sorted set).

encoding

Specifies the internal encoding (e.g., int, embstr, raw for strings; ziplist or linkedlist for lists).

lru

Tracks the last access time, used by eviction policies.

refcount

Reference count for memory reclamation; objects with refcount > 1 are shared (e.g., integer strings 0‑9999).

4.4 Simple Dynamic String (SDS)

SDS replaces C strings in Redis. It stores buf , len , and free fields, providing O(1) length retrieval, automatic resizing, and safe binary data handling.

SDS vs C string

Length retrieval: O(1) vs O(n).

Buffer overflow protection.

Efficient reallocation on growth/shrink.

Binary data support.

5. Redis Object Types and Internal Encodings

Each of the five object types has at least two possible encodings, allowing Redis to choose the most memory‑efficient representation based on size and count.

5.1 String

Encodings: int (8‑byte integer), embstr (≤39 bytes, stored contiguously), raw (>39 bytes). Transition from embstr to raw occurs on modification.

5.2 List

Encodings: ziplist (compact, for ≤512 elements and each ≤64 bytes) or linkedlist (doubly linked list). Conversion is one‑way from ziplist to linkedlist.

5.3 Hash

Internal encodings: ziplist (small hash) or hashtable . Conversion follows the same size/length thresholds as lists.

5.4 Set

Encodings: intset (all integer elements, ≤512) or hashtable . Transition is one‑way from intset to hashtable.

5.5 Sorted Set

Encodings: ziplist (≤128 elements, each ≤64 bytes) or skiplist . Conversion is one‑way from ziplist to skiplist.

6. Practical Applications

6.1 Estimating Memory Usage

For 90 000 string key‑value pairs (key length = 7 bytes, value length = 7 bytes, non‑integer), each dictEntry occupies 80 bytes after jemalloc rounding, and the bucket array (size = 2^17 = 131 072) adds 1 048 576 bytes. Total ≈ 8 248 576 bytes, matching experimental results.

6.2 Optimizing Memory Consumption

Leverage jemalloc size classes: reducing key length from 8 bytes to 7 bytes halves allocation from 32 bytes to 16 bytes.

Prefer integer types over strings when possible (int encoding saves space).

Increase the shared integer pool (default 0‑9999) to reduce duplicate objects.

Avoid over‑engineering for small datasets; optimization benefits become significant at millions of keys.

6.3 Monitoring Fragmentation Ratio

A healthy mem_fragmentation_ratio is around 1.03 for jemalloc. Ratios > 1 indicate fragmentation; ratios < 1 suggest swapping and the need for more physical memory or data reduction.

Setting appropriate maxmemory policies (e.g., volatile‑lru, allkeys‑lru) helps Redis evict the least recently used keys when memory limits are reached.