Understanding and Mitigating Bigkey Issues in Redis Operations

During Redis operation, the presence of bigkeys—keys whose size exceeds 1 MB for strings or contains more than 2 000 elements for other data structures—can severely degrade response time and even cause availability loss.

Background: In a large‑scale environment with over 2 200 Redis clusters and 45 000 instances, a full‑network bigkey scan would take years, prompting the need for more efficient solutions.

Definition and Causes: Bigkeys arise from improper program design or unexpected data growth, commonly in three scenarios: statistical keys that accumulate user IPs, cache keys that store large serialized objects, and queue keys that grow when consumption lags.

Harms: Bigkeys lead to uneven memory distribution, timeout blocking (because Redis processes commands single‑threaded), network congestion (large payloads per request), and migration difficulties during horizontal scaling, often causing migration timeouts or master‑slave failovers.

Detection Methods: Two main approaches are used: (1) the --bigkeys scan command (preferably on a slave) which reports the top‑1 bigkey per data type, and (2) RDB‑file analysis with tools like rdb‑tools to extract the largest keys. Both methods can be integrated into the Daas platform.

Optimization Strategies: (1) Split bigkeys by reducing string length or breaking large collections into multiple keys (e.g., list1, list2… or hash%100, or date‑based keys). (2) Enhance analysis tools to run concurrently across slaves with configurable concurrency and pause/resume capabilities. (3) Adjust migration parameters—extend cluster-node-timeout to 15 minutes, limit migrate timeout to 10 seconds with three retries and improved logging—to minimize blocking and quickly locate offending keys.

Conclusion: Effective bigkey management requires source‑level prevention, timely detection through inspection and analysis tools, and systematic remediation such as key splitting and migration tuning, thereby improving Redis stability and overall service availability.