How JIMDB’s Big‑Hot Key Strategy Boosts Cache Performance by 80%

JIMDB is a Redis‑based distributed cache service offering high performance, high availability, and online scaling, used across JD.com business units. This article focuses on the optimization and insights around "Big‑Hot Key" issues.

Challenge

High‑performance distributed caches have long suffered from "Big Key" and "Hot Key" problems that can exhaust resources, increase latency, and trigger avalanche failures. Traditional static‑threshold definitions and passive mitigation are inadequate for modern, high‑throughput workloads.

Innovation

JIMDB proposes the "Big‑Hot Key" concept, shifting the definition from static key attributes to real‑time resource impact (CPU and network bandwidth), enabling precise identification of operations that actually pressure the service.

Solution

A comprehensive, multi‑layer proactive governance framework is built. Its core is a sophisticated server‑side identification engine that actively and in real time discovers risky keys, complemented by automated and manual intervention tools such as server‑side auto‑caching, command‑request circuit breaking, blacklist mechanisms, and an intelligent client‑side cache that guarantees data consistency.

Results

Online tests show the solution reduces instance CPU usage from 100% overload to about 30% and raises sustainable OPS from roughly 6,700 to nearly 12,000, an ~80% performance improvement that markedly lowers critical stability risks.

Significance

The approach marks a paradigm shift from a passive, manual incident‑response model to an embedded, proactive, highly automated governance model, reducing operational burden and enhancing system resilience.

1. Redefining Performance Bottlenecks: The Big‑Hot Key Paradigm

Traditional "Big Key" (large value size) and "Hot Key" (high access frequency) definitions describe risks from data volume and request rate separately. However, many real‑world failures involve keys that do not meet either static threshold yet still cause severe resource consumption when specific commands are executed. "Big‑Hot Key" captures this by considering the combined effect of data size and access frequency on CPU and bandwidth.

Big Key : Value size >1 MB or collection elements >5,000, leading to memory pressure, operation blocking, and persistence delays.

Hot Key : QPS >10,000, causing single‑shard bottlenecks, cache breakdown, and network congestion.

Big‑Hot Key : Any key‑command pair whose cumulative data processing and frequency exhaust CPU or bandwidth, even if the key itself is not large or hot by traditional metrics.

2. Intelligent Perception Layer: Multi‑Dimensional Identification Engine

The engine follows a three‑stage waterfall detection principle, prioritizing low‑cost checks:

Bandwidth bottleneck detection – compare instantaneous bandwidth of a command against 70% of the instance’s total output bandwidth.

Collection size bottleneck – static checks for element count >100 k or response size >1 MB.

CPU bottleneck – machine‑learning‑based prediction (polynomial regression + linear interpolation) comparing actual QPS with predicted QPS limits.

2.1 Bandwidth Bottleneck Detection

The engine estimates the response size of a command and its execution frequency within a short window to compute instantaneous bandwidth. If the bandwidth exceeds 70% of the instance’s output limit, the key is flagged as a Big‑Hot Key.

2.2 Collection Size Static Risk Pre‑Check

For collection types (List, Hash, Set, ZSet), if element count exceeds 100 k or a single command returns >1 MB, the operation is directly marked as a Big‑Hot Key with a fixed high‑priority index (INT_MAX).

2.3 Machine‑Learning CPU Prediction

When the first two checks pass, the engine predicts the maximum sustainable QPS for the specific command using a hybrid model of polynomial regression and linear interpolation. If actual QPS surpasses the predicted limit, the key is identified as a Big‑Hot Key, and its index grows quadratically with the degree of overload.

3. Multi‑Layer Defense: Big‑Hot Key Governance Suite

Accurate identification is only the first step; comprehensive mitigation ensures system stability.

3.1 Server‑Side Automatic Mitigation

Automatic caching : When a read‑intensive Big‑Hot Key is detected, the server caches the serialized response. Subsequent identical requests hit the cache, bypassing expensive traversal and serialization, drastically reducing CPU load.

Circuit breaker (default off) : For destructive access patterns, the breaker can reject all requests to the offending key, returning an error message, thus protecting the instance from overload.

3.2 Intelligent Client Collaboration

The SDK caches the value and a version number on first access. Subsequent requests send only the key and cached version; the server replies with a short confirmation if the version matches, otherwise returns the fresh value and new version. This guarantees strong consistency while cutting network traffic.

3.3 Manual Intervention Tools

Manual blacklist : Operators can add keys or commands to a blacklist via the management console, providing a flexible, controllable way to block problematic traffic.

High‑availability management port : An independent management thread listens on a separate port, allowing diagnostics and control even when the main thread is blocked.

3.4 Productized Emergency Plans

A one‑click emergency plan aggregates key logs, slow logs, network captures, and offers immediate actions such as circuit breaking, dramatically shortening MTTR.

4. Empirical Evaluation: Quantitative Impact in Production

Tests on physical machines (8 CPU, 16 GB RAM) compare a baseline version without Big‑Hot Key features to the optimized version with auto‑identification and caching.

CPU usage : Reduced from 100% to ~30% after the engine activates.

OPS : Increased from ~6,700 to ~12,000, an ~80% gain.

Stability : No alerts or crashes after optimization.

Overall, the solution cuts CPU consumption by ~70 percentage points, doubles throughput, and transforms an unstable system into a stable, high‑performance service.