Unlock Redis Performance: Deep Dive into Its Memory Model and Optimization

1. Introduction

Redis is one of the hottest in‑memory databases. By reading and writing data directly in memory it dramatically speeds up I/O, making it indispensable for high‑concurrency websites. Redis provides five object types (string, hash, list, set, sorted set) whose characteristics are essential for understanding its memory model.

2. Redis Memory Statistics

After connecting with redis-cli, the info memory command displays memory‑related information only.

Key fields:

used_memory : total memory allocated by the allocator (bytes), includes virtual memory.

used_memory_rss : memory the Redis process occupies in the OS (bytes), matches top / ps values.

mem_fragmentation_ratio : used_memory_rss / used_memory, indicates fragmentation.

mem_allocator : allocator used (libc, jemalloc, tcmalloc); default is jemalloc.

When mem_fragmentation_ratio exceeds 1, fragmentation is high; values around 1.03 are healthy for jemalloc. Ratios below 1 suggest heavy swapping.

3. Redis Memory Partitioning

Memory consumption is divided into:

3.1 Data

Stored in used_memory. Redis stores values using five types, each with multiple internal encodings, wrapped in redisObject and SDS.

3.2 Process Overhead

The main process consumes a few megabytes for code and constant pools; this memory is not allocated by jemalloc and is not counted in used_memory. Child processes (AOF/RDB rewrite) also use memory but are excluded from the statistics.

3.3 Buffers

Client buffers, replication backlog, and AOF buffers are allocated by jemalloc and therefore appear in used_memory.

3.4 Fragmentation

Fragmentation arises from allocation/deallocation patterns and the allocator design. Jemalloc reduces fragmentation; a restart can defragment memory by reloading data.

4. Details of Redis Data Storage

4.1 Overview

Key concepts include the memory allocator (jemalloc), Simple Dynamic String (SDS), the five object types, their internal encodings, and redisObject. The diagram for SET hello world shows the data flow.

Components:

dictEntry : stores pointers to key and value.

Key : stored as an SDS structure.

redisObject : wraps the value, indicating its type via the type field and pointing to the actual data via ptr.

jemalloc : allocates memory for dictEntry, SDS, and redisObject.

4.2 jemalloc

Jemalloc is the default allocator. It divides memory into small, large, and huge classes, further split into fixed‑size chunks. For example, a 130‑byte object is placed in a 160‑byte chunk.

4.3 redisObject

All Redis objects are stored through redisObject, which holds fields such as type, encoding, lru, refcount, and ptr. The structure occupies 16 bytes (4 bit + 4 bit + 24 bit + 4 Byte + 8 Byte = 16 Byte).

4.4 SDS (Simple Dynamic String)

Redis uses SDS instead of C strings. SDS stores len, free, and a buffer. Benefits include O(1) length retrieval, automatic reallocation to avoid buffer overflow, efficient binary data handling, and reduced memory fragmentation.

5. Redis Object Types and Internal Encodings

Each of the five object types has at least two internal encodings, allowing Redis to choose the most space‑efficient representation.

5.1 String

int : 8‑byte integer.

embstr : ≤ 39 bytes, stored as a single contiguous allocation.

raw : > 39 bytes, allocated separately.

5.2 List

Encodings: ziplist (compressed list) for small lists, or linkedlist for larger ones. Conversion occurs when element count exceeds 512 or any element exceeds 64 bytes.

5.3 Hash

Internal encodings: ziplist for small hashes, hashtable otherwise. Conversion criteria are the same as for lists.

5.4 Set

Encodings: intset for small integer sets, otherwise hashtable.

5.5 Sorted Set

Encodings: ziplist for ≤ 128 elements with short members, otherwise skiplist.

6. Practical Applications

6.1 Estimating Memory Usage

For 90 000 string key‑value pairs (7‑byte keys and values, non‑integer), each entry occupies 80 bytes (dictEntry 32 + key 16 + redisObject 16 + value 16). The bucket array size is the next power of two (131 072) × 8 bytes. Total ≈ 8.25 MB, matching experimental results.

6.2 Optimizing Memory Consumption

Leverage jemalloc size classes: reducing key length from 8 bytes to 7 bytes halves the allocation size.

Prefer integer types over strings when possible; integers use 8 bytes.

Increase REDIS_SHARED_INTEGERS to share more integer objects.

Avoid over‑engineering for small datasets; the memory saved may be negligible.

6.3 Monitoring Fragmentation Ratio

A ratio around 1.03 is normal. Values > 1 indicate fragmentation; consider restarting Redis to defragment. Ratios < 1 imply swapping; increase physical memory or reduce data volume, and configure an appropriate maxmemory‑policy.

Original source: http://www.cnblogs.com/kismetv/p/8654978.html