How to Mitigate Redis Performance Issues Caused by Large Keys

Problem Overview

If a Redis key holds a very large value, the single‑threaded nature of Redis can cause noticeable performance degradation; queries or deletions of such keys may block the server.

Splitting Large Keys

To avoid the bottleneck, split the large object into multiple K‑V pairs and retrieve them with MULTIGET. This spreads I/O pressure across several keys and, if needed, across multiple Redis instances.

When the whole object must be stored and retrieved as a unit.

When only part of the object is accessed each time.

Both cases can be handled by breaking the object into several keys.

Using Redis Hashes

An alternative is to store the large object in a Redis HASH, where each field represents a specific attribute. Use HGET or HMGET to fetch partial values, and HSET / HMSET to update them.

Memory Overhead in Cluster Mode

When the cluster stores a massive number of keys, memory consumption rises due to:

Each key’s own space, including its prefix.

Slot‑to‑key mapping tables required by the cluster, whose pointers become a large waste when keys are numerous.

With billions of keys, this overhead can be severe.

Bucket‑Based Hash Design

Reduce the number of top‑level keys by grouping them into hash buckets. For example, estimating 200 million keys and storing 100 fields per hash yields about 2 million buckets. Compute a bucket index with a hash function: hash(123456789) % 200W Ensure the hash result is non‑negative; otherwise adjust the modulo operation.

When storing, call HSET(bucketKey, field, value); when reading, use HGET(bucketKey, field). This approach keeps memory usage low while preserving fast access.