Understanding and Detecting BigKey and HotKey Issues in Redis Clusters

In daily production, improper access to Redis clusters often leads to online problems such as BigKey and HotKey issues. These problems not only degrade service performance but also affect normal user functionality and can cause widespread service failures, sometimes with cascading effects.

BigKey (large key) refers to a Redis key that occupies an excessively large amount of memory. Redis stores data as key‑value pairs, and each data structure (String, List, Hash, etc.) has its own size limits. When a key’s content approaches or exceeds these limits, a BigKey is created. For example, a List storing 100 MB of data in a Redis instance with only 150 MB of total memory can cause request blocking due to Redis’s single‑threaded nature, leading to timeouts and 404 errors.

HotKey (hot key) occurs when a single key on a Redis shard receives a massive number of requests in a short period, causing that shard to become a hotspot. This results in resource imbalance across the cluster. An illustrative example is a celebrity’s social‑media account that suddenly receives a surge of traffic, overwhelming the cache key that stores the account data.

Problems caused by BigKey include:

Data request timeouts: because Redis processes requests sequentially, a large key can block subsequent requests.

Bandwidth congestion: repeatedly fetching a large key consumes significant network bandwidth.

Memory overflow or processing blockage: continuous growth of a large key can lead to memory exhaustion or long deletion times, affecting master‑slave synchronization.

Problems caused by HotKey include:

Shard service collapse: excessive requests to a single shard can cause that shard to fail.

Weakening of Redis cluster advantages: uneven request distribution reduces the benefits of a distributed cache.

Potential financial loss: in extreme cases, delayed data processing can cause monetary damage.

Cache breakdown: concentrated requests may bypass the cache and overload the underlying database.

High CPU usage: a hot shard can monopolize CPU resources, impacting other services.

How to discover BigKey and HotKey during testing :

1. Business analysis combined with technical solutions: analyze scenarios such as shopping carts without size limits or activity qualification lists that may accumulate large amounts of data, and assess request patterns and QPS.

2. Use Redis commands to identify problematic keys:

redis-cli -a password --bigkeys

redis-cli -a password --hotkeys

3. Leverage tools:

• Visual Redis clients (e.g., Another Redis Desktop Manager) to view cluster information and spot anomalies.

• Open‑source utilities such as redis‑rdb‑tools for deeper analysis.

These methods help engineers quickly detect and mitigate BigKey and HotKey issues, ensuring stable Redis performance.

References :

Redis command reference documentation.

GitHub: https://github.com/sripathikrishnan/redis-rdb-tools

Another Redis Desktop Manager download page.