How JD’s Hotkey Framework Detects and Mitigates Hot Data in Milliseconds

Project Introduction

JD App backend middleware detects hot data at millisecond precision and pushes the hot keys to server‑cluster memory, greatly reducing the query pressure of hot keys on the data layer.

It can detect any sudden hot requests—such as a surge of requests for the same product, crawler or bot users, and heavily accessed interfaces—then push these hot keys to all JVMs of the application, allowing clients to decide how to handle them (e.g., local cache, reject access, circuit‑break, or return default values). The hot keys remain consistent across the entire application cluster.

Core Function : Hot data detection and push to all servers.

Applicable Scenarios :

MySQL hot data local cache

Redis hot data local cache

Blacklist user local cache

Crawler user rate limiting

Interface and user dimension rate limiting

Single‑machine interface/user rate limiting

Cluster user dimension rate limiting

Cluster interface dimension rate limiting

Performance Metrics

The framework has been stress‑tested with two main performance indicators:

Detection performance: An 8‑core worker can handle 160,000 key detection tasks per second; a 16‑core worker processes at least 300,000 per second, with actual tests reaching 370,000 while keeping CPU stable and no anomalies.

Push performance: Under high‑concurrency writes, the framework pushes steadily at 100,000‑120,000 times per second. For example, with 1,000 servers and each worker generating 100 hot keys per second, the system pushes 100,000 times per second and delivers all within 1 s. In pure push mode, it can sustain 400,000‑600,000 pushes per second, with an 800,000‑push peak for a few seconds.

Real‑World Validation

The hotkey framework has undergone multiple high‑pressure tests and survived JD’s 2020 618 and Double‑11 promotions.

During operation, billions of keys are detected daily, accurately capturing many crawler and bot users as well as popular products, and pushing them to each service’s memory in milliseconds, which significantly reduces the load on the data storage layer and boosts application performance.

During peak promotions, the worker cluster achieved a throughput of 15 million hot keys per second, and the locally cached hot keys accounted for over 50 % of total accesses, halving the load on Redis.

Architecture Design

The framework does not rely on any custom components and is independent of Redis; it relies on Netty connections. Clients send keys to be measured, workers perform distributed computation, and hot keys are pushed back to clients, making the system lightweight.

The framework consists of four parts:

etcd Cluster : Serves as a high‑performance configuration center with minimal resource usage, providing efficient watch/subscribe services. It stores rule configurations, worker IPs, detected hot keys, and manually added hot keys.

Client‑side JAR : Added as a dependency in services, it enables easy hot‑key checks, reports keys, listens for rule changes, worker updates, and hot‑key updates, and maintains a local Caffeine cache.

Worker Cluster : Independent Java programs that connect to etcd, periodically report their IPs, receive keys from clients, aggregate counts, and push hot keys to clients once configured thresholds are met.

Dashboard Console : A Java program with a visual UI that connects to etcd, allows setting key detection rules (e.g., 20 occurrences within 2 seconds), listens for hot‑key events, stores records, and supports manual addition or removal of hot keys for client consumption.

Project Address

https://gitee.com/jd-platform-opensource/hotkey