Master Redis Memory: 7 Proven Techniques to Store More Data with Less RAM

Redis Storage Fundamentals

Redis stores data in a redisDb structure. The key field is dict, a global hash table holding key‑value pairs. Other fields: expires (expiration), blocking_keys / ready_keys (blocking commands), watched_keys (WATCH), id (database ID), clusterSlotToKeyMapping (cluster slot mapping).

The hash table uses two arrays ht_table[2] to support incremental rehashing. Each bucket is a dictEntry containing a pointer to the key, a union for the value, and a next pointer for collision chains.

struct dict {
    dictType *type;
    dictEntry **ht_table[2];
    unsigned long ht_used[2];
    long rehashidx;
    int16_t pauserehash;
    signed char ht_size_exp[2];
};

Each dictEntry points to a redisObject that records object type, encoding, LRU/LFU information, reference count, and a pointer to the actual data.

typedef struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS;
    int refcount;
    void *ptr;
} robj;

1. Key‑Value Pair Optimization

Shorten keys (e.g., users:friends:996 → u:fs:996) to reduce memory.

Remove unnecessary fields and store compact representations (e.g., numeric status codes).

Compress large values with GZIP, Snappy, or use binary serialization libraries such as Protostuff or Kryo instead of plain JSON.

2. Small Collection Encoding Optimization

Redis selects internal encodings based on element count and size. Common encodings:

String : int (small integers), embstr (≤44 bytes), raw (larger strings).

List : ziplist for small lists, linkedlist (pre‑3.0), quicklist (default, hybrid of ziplist and linkedlist).

Set : intset for integer sets, otherwise a hash table.

Hash : ziplist when entry count < hash-max-ziplist-entries and value size < hash-max-ziplist-value; otherwise a hash table.

Sorted Set : ziplist under thresholds, otherwise a skiplist.

Default thresholds (redis.conf):

hash-max-ziplist-entries 512
hash-max-ziplist-value 64
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
set-max-intset-entries 512

3. Object Shared Integer Pool

At startup Redis creates a pool of pre‑allocated integer objects for values 0‑9999. When a value fits in this range Redis reuses the same object, saving memory. The pool is ineffective if maxmemory with an LRU policy is enabled (LRU needs distinct timestamps) or when ziplist‑encoded collections store integers.

4. Bit‑Level Operations (Bitmap)

For binary state tracking (e.g., login status, daily sign‑in) use a Bitmap. Each bit represents a 0/1 state, giving eight states per byte and dramatically reducing memory compared with separate strings.

5. Hash Type Consolidation

Store related attributes of an entity in a single hash instead of separate keys, e.g.:

HSET user:123 name "Alice" age 30

This reduces the number of redisObject instances and cuts memory usage.

6. Memory Fragmentation Optimization

Fragmentation arises from OS allocation granularity and frequent key updates/deletions. In Redis <4.0 the only remedy is a restart (RDB reload or master‑slave failover). From Redis 4.0 onward you can:

Manually run MEMORY PURGE to free fragmented memory.

Enable automatic defragmentation with CONFIG SET activedefrag yes (or activedefrag yes in redis.conf).

Configure thresholds: active-defrag-ignore-bytes 200mb and active-defrag-threshold-lower 6 (percentage of fragmentation).

Control CPU usage with active-defrag-cycle-min 15 and active-defrag-cycle-max 50.

7. Using 32‑Bit Redis

A 32‑bit Redis process uses smaller pointers, reducing per‑key overhead, but limits addressable memory to ~4 GB. Deploy multiple 32‑bit instances in a cluster to scale out. RDB/AOF files are architecture‑agnostic, so a 64‑bit instance can restore a 32‑bit backup and vice‑versa.

References

https://redis.io/docs/reference/optimization/memory-optimization/

《Redis 核心技术与实战》

https://segmentfault.com/a/1190000041771534