Redis Memory Management and Optimization Practices

Redis is an in‑memory key‑value database where memory plays a central role; all operations revolve around it. Users often ask how to store data efficiently, reduce costs, and troubleshoot memory alarms.

The article first analyzes Redis's memory model, describing the main memory metrics:

used_memory : total memory allocated by Redis (default jemalloc), including internal data structures, objects, caches, and Lua scripts.

used_memory_rss : memory as seen by the operating system.

Memory fragmentation : difference between allocated and actually used memory, reported by mem_fragmentation_ratio.

It then breaks down the memory consumption of different object types (String, List, Set, Hash, Zset, Stream) and their internal encodings (int, embstr, raw, ziplist, linkedlist, quicklist, intset, hashtable, skiplist). The choice of encoding depends on size thresholds and configuration parameters.

Client‑side buffers (normal, replica, pub/sub) and AOF buffers are also covered. The client-output-buffer-limit setting controls the maximum size of each client’s output buffer, while repl-backlog-size defines the replication backlog.

Memory fragmentation is explained as a side‑effect of jemalloc’s size classes and the copy‑on‑write mechanism. Automatic and manual defragmentation can be enabled with parameters such as activedefrag yes, active-defrag-ignore-bytes, and related thresholds.

Sub‑process memory usage is discussed, noting that Redis forks child processes for AOF rewrite and RDB generation, relying on Linux's copy‑on‑write. The article warns against enabling Transparent Huge Pages (THP) because they increase the copy granularity.

Two practical case studies illustrate the concepts:

Client buffer abnormal growth : A cluster showed rapid memory increase without new writes. Investigation revealed a single client with an enormous output buffer caused by large GET commands and pipeline usage. The solution involved limiting the client output buffer with

config set client-output-buffer-limit normal 4096mb 2048mb 120

and advising the application to avoid excessive pipelining.

Replica memory explosion : Some replica nodes exceeded memory limits while the master remained stable. The root cause was frequent APPEND operations that triggered repeated reallocations on replicas, leading to higher memory usage than the master. Adjustments included expanding replica memory, monitoring memory usage, and keeping master‑replica memory limits consistent.

The article concludes that understanding Redis's internal memory allocation enables faster diagnosis of memory anomalies and more effective performance tuning, emphasizing key design practices such as keeping keys small, avoiding large ziplist encodings for frequently updated data, and setting appropriate buffer limits.

Reference: "Redis Design and Implementation" book.