Why Redis Used Memory Jumped to 78 GB and How to Diagnose It

Background

A production Redis instance experienced a memory spike: used_memory reached 78.9 GB while the configured maxmemory was only 16 GB, causing extensive data eviction.

How INFO’s used_memory is generated

When the INFO command is executed, Redis calls genRedisInfoString (in server.c). This function obtains the value from zmalloc_used_memory() and prints it as used_memory and related fields.

#include <server.c>
size_t zmalloc_used_memory(void) {
    size_t um;
    atomicGet(used_memory, um);
    return um;
}

What is used_memory

used_memory

is a static variable of type redisAtomic size_t (an alias for the C11 _Atomic keyword). It is updated atomically via two macros:

#define update_zmalloc_stat_alloc(__n) atomicIncr(used_memory, (__n))
#define update_zmalloc_stat_free(__n)  atomicDecr(used_memory, (__n))

Memory allocation functions such as ztrymalloc_usable call update_zmalloc_stat_alloc, while zfree calls update_zmalloc_stat_free, ensuring the counter stays accurate under concurrency.

Typical memory consumption breakdown

used_memory

consists of two main parts:

Data itself – reported as used_memory_dataset in INFO.

Overhead – reported as used_memory_overhead, covering internal data structures, replication buffers, client buffers, AOF buffers, Lua script caches, etc.

The overhead is calculated by getMemoryOverheadData() (in object.c), which aggregates memory from server startup, replication backlog, client memory, AOF buffers, Lua caches, and per‑database hash tables.

#include <object.c>
struct redisMemOverhead *getMemoryOverheadData(void) {
    size_t mem_total = 0;
    size_t zmalloc_used = zmalloc_used_memory();
    // … accumulate various components …
    mh->overhead_total = mem_total;
    mh->dataset = zmalloc_used - mem_total;
    return mh;
}

Changes in Redis 7

Redis 7 adds new overhead items such as cluster_links, functions_caches, and slot‑to‑key hash‑table memory. It also introduces Multi‑Part AOF, removing the separate AOF rewrite buffer.

Most notably, Redis 7 replaces per‑slave replication buffers with a single global replication buffer. The structure replBufBlock stores a block of this shared buffer, and a reference count tracks how many replicas use it.

typedef struct replBufBlock {
    int refcount;   /* Number of replicas or repl backlog using. */
    long long id;   /* Unique incremental number. */
    long long repl_offset; /* Start replication offset of the block. */
    size_t size, used;
    char buf[];
} replBufBlock;

The new memory‑calculation logic distinguishes between the global buffer size and the portion attributed to slave clients:

if (listLength(server.slaves) && (long long)server.repl_buffer_mem > server.repl_backlog_size) {
    mh->clients_slaves = server.repl_buffer_mem - server.repl_backlog_size;
    mh->repl_backlog = server.repl_backlog_size;
} else {
    mh->clients_slaves = 0;
    mh->repl_backlog = server.repl_buffer_mem;
}
/* Add Rax tree overhead for the global buffer */
if (server.repl_backlog) {
    mh->repl_backlog += server.repl_backlog->blocks_index->numnodes * sizeof(raxNode) +
                        raxSize(server.repl_backlog->blocks_index) * sizeof(void*);
}

When does eviction occur?

Eviction is not triggered merely because used_memory exceeds maxmemory. The following conditions must be met: maxmemory must be greater than zero. maxmemory-policy cannot be noeviction.

The memory considered for eviction is used_memory - mem_not_counted_for_evict. Eviction happens only when this value exceeds maxmemory.

The function freeMemoryGetNotCountedMemory() computes mem_not_counted_for_evict by summing the memory of slave replication buffers, AOF buffers, and AOF rewrite buffers.

size_t freeMemoryGetNotCountedMemory(void) {
    size_t overhead = 0;
    // slave replication buffers
    // AOF buffers
    // AOF rewrite buffers
    return overhead;
}

Memory‑analysis script

A Python script ( redis_mem_usage_analyzer.py) periodically runs INFO, parses the output, and prints a concise table showing the growth of each memory component. The script also supports a --client flag to list per‑client memory usage (input buffer Qbuf, output buffer Omem, and total memory).

# python3 redis_mem_usage_analyzer.py -host 10.0.1.182 -p 6379
Metric(2024-09-12 04:52:42)   Old Value    New Value(+3s)   Change per second
==========================================================================================
Summary
---------------------------------------------
used_memory               16.43G      16.44G      1.1M
used_memory_dataset       11.93G      11.93G      22.66K
used_memory_overhead      4.51G       4.51G       1.08M
...
Evict & Fragmentation
---------------------------------------------
maxmemory                 20G        20G        0B
mem_not_counted_for_evict 458.45M    461.73M    1.1M
mem_counted_for_evict     15.99G     15.99G     2.62K
...

Script source: https://github.com/slowtech/dba-toolkit/blob/master/redis/redis_mem_usage_analyzer.py