Master Redis Memory: 7 Proven Techniques to Store More Data with Less RAM

Redis Storage Fundamentals

Redis stores data in a central redisDb structure (source: https://github.com/redis/redis/blob/7.0/src/server.h). The core component is the global hash table dict, which holds all key‑value pairs with O(1) lookup.

Definition of dict (source: https://github.com/redis/redis/blob/7.0/src/dict.h):

struct dict {
    dictType *type;
    dictEntry **ht_table[2];
    unsigned long ht_used[2];
    long rehashidx;
    int16_t pauserehash;
    signed char ht_size_exp[2];
};

Each bucket in ht_table stores a dictEntry:

typedef struct dictEntry {
    void *key;
    union {
        void *val;
        uint64_t u64;
        int64_t s64;
        double d;
    } v;
    struct dictEntry *next;
} dictEntry;

Keys are always SDS strings; values are pointers to redisObject (source: https://github.com/redis/redis/blob/7.0/src/server.h):

typedef struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS;
    int refcount;
    void *ptr;
} robj;

The type field identifies the logical data type (string, list, set, hash, zset, etc.), while encoding specifies the concrete in‑memory representation.

1. Key‑Value Pair Optimisation

Shorten keys : use concise prefixes, e.g. u:fs:996 instead of users:friends:996.

Trim values : drop unnecessary fields, replace verbose strings with small integers, and compress payloads with GZIP or Snappy.

Efficient serialization : prefer binary serializers (protobuf, Kryo, etc.) over Java default serialization; compressed JSON can reduce size by ~60%.

2. Small‑Collection Encoding Optimisation

Redis automatically selects compact encodings for small collections. Default thresholds for Redis 7.0 are:

hash-max-ziplist-entries 512
hash-max-ziplist-value 64
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
set-max-intset-entries 512

String : int (≤20 digits), embstr (≤44 bytes), otherwise raw.

List : ziplist when element count < hash-max-ziplist-entries and element size < hash-max-ziplist-value; otherwise quicklist (default since 3.2).

Set : intset for pure integers below set-max-intset-entries; otherwise a hash table.

Hash : ziplist under the same thresholds as list; otherwise a hash table.

Sorted Set : ziplist for tiny sets; otherwise a skiplist.

3. Object Shared Pool

At startup Redis pre‑allocates redisObject instances for integers 0‑9999, allowing many keys to share the same object and save memory. The pool is ineffective when LRU eviction or ziplist‑encoded sets require distinct integer objects.

4. Bit‑Level Operations

For binary state tracking (e.g., login status, daily sign‑in), use a Bitmap built on the SDS string structure. Each bit represents a user’s state, compressing 8 states into one byte.

5. Hash Type Optimisation

Group related fields of an entity into a single hash instead of separate string keys. Example:

hset users:shenzhen:999 name "码哥"
 hset users:shenzhen:999 age 18
 hset users:shenzhen:999 hobby "coding"

This reduces the number of redisObject allocations and associated metadata.

6. Memory Fragmentation Optimisation

Fragmentation arises from OS allocation granularity and frequent key updates/deletions. Redis 4.0+ provides defragmentation:

Manual : memory purge Automatic : set activedefrag yes in redis.conf or via CONFIG SET activedefrag yes.

Trigger thresholds: active-defrag-ignore-bytes 200mb and active-defrag-threshold-lower 6 (percent of total memory).

CPU limits: active-defrag-cycle-min 15 and active-defrag-cycle-max 50.

7. Using 32‑Bit Redis

Running Redis as a 32‑bit binary reduces pointer size, cutting per‑key overhead, but caps the process memory at ~4 GB. Clustering multiple 32‑bit nodes overcomes this limit, and RDB/AOF files are portable between 32‑ and 64‑bit instances.

References

Redis Memory Optimisation Guide: https://redis.io/docs/reference/optimization/memory-optimization/

《Redis 核心技术与实战》

SegmentFault article on Redis memory tricks: https://segmentfault.com/a/1190000041771534