Unlocking Redis: How Its Memory Model Impacts Performance and Cost

Introduction

Redis is one of the most popular in‑memory databases. It stores data using five object types (string, hash, list, set, sorted set) and its memory model is crucial for estimating memory consumption, optimizing usage, and troubleshooting issues.

Redis Memory Statistics

After connecting with redis-cli, the INFO memory command shows memory‑related metrics:

used_memory : total memory allocated by Redis (including virtual memory).

used_memory_rss : memory occupied by the Redis process as seen by the OS.

mem_fragmentation_ratio : used_memory_rss / used_memory, indicating fragmentation level.

mem_allocator : the allocator used (libc, jemalloc, or tcmalloc; default is jemalloc).

Redis Memory Partition

Memory consumption can be divided into:

Data : the key‑value pairs stored in used_memory.

Process memory : code, constant pools, etc., usually a few megabytes and not counted by jemalloc.

Buffer memory : client buffers, replication backlog, AOF buffer (allocated by jemalloc and counted in used_memory).

Memory fragmentation : unused space left by the allocator.

Data Storage Details

When a SET hello world command is executed, the following structures are involved:

dictEntry : stores pointers to the key and value.

Key : stored as a Simple Dynamic String (SDS).

redisObject : wraps the value and records its type, encoding, LRU, reference count, and a pointer to the actual data.

jemalloc : allocates memory for dictEntry, redisObject, and SDS structures.

jemalloc

Redis uses jemalloc by default because it reduces fragmentation. jemalloc divides memory into small, large, and huge classes and allocates the smallest fitting block (e.g., a 130‑byte object is placed in a 160‑byte block).

redisObject Fields

type : 4‑bit field indicating the object type (string, list, hash, set, sorted set).

encoding : 4‑bit field specifying the internal encoding (e.g., int, embstr, raw for strings).

lru : records the last access time, used by LRU eviction policies.

refcount : reference count; objects with refcount > 1 are shared (currently only integer strings).

ptr : pointer to the actual data (e.g., an SDS for a string value).

SDS (Simple Dynamic String)

SDS adds len and free fields to a C string, providing O(1) length retrieval, automatic buffer resizing, and safe binary data handling. Its layout is buf + len + free + 1 bytes.

Object Types and Internal Encodings

String

Three encodings: int: 8‑byte integer. embstr: ≤39‑byte strings stored in a single contiguous allocation. raw: >39‑byte strings stored separately.

List

Encodings: ziplist: compact representation used when the list has < 512 elements and each element < 64 bytes. linkedlist: doubly linked list for larger lists.

Hash

Encodings: ziplist: used when the hash has < 512 fields and each field/value < 64 bytes. hashtable: default for larger hashes.

Set

Encodings: intset: stores integer elements when the set has < 512 members and all are integers. hashtable: otherwise.

Sorted Set

Encodings: ziplist: when < 128 elements and each member < 64 bytes. skiplist: otherwise.

Practical Applications

Estimating Memory Usage

For 90 000 string key‑value pairs (key length = 7 bytes, value length = 7 bytes, using embstr), each dictEntry consumes 80 bytes and the bucket array (size = 131 072) adds 1 048 576 bytes, resulting in an estimated 8 248 576 bytes.

Optimizing Memory Consumption

Leverage jemalloc’s size classes (e.g., reducing key length from 8 bytes to 7 bytes halves the allocation).

Prefer integer types over strings when possible.

Increase REDIS_SHARED_INTEGERS to share more integer objects.

Monitoring Fragmentation

A fragmentation ratio around 1.03 is healthy. Ratios > 1 indicate excess fragmentation; ratios < 1 suggest swapping and require more physical memory or data reduction.