How JD-hotkey Achieves Million‑Level Hot Data Detection with Millisecond Latency

JD-hotkey is a backend hot data detection framework used in JD APP, proven through high‑pressure stress tests and the 2020 JD 618 promotion.

During operation it processes billions of keys daily, accurately capturing crawlers, abusive users, and hot items, and pushes hot keys to service memory within milliseconds, greatly reducing query pressure on the data layer and improving application performance.

Performance Metrics

Detection performance: An 8‑core worker can handle 160,000 key detection tasks per second; a 16‑core worker processes over 300,000 per second, with real‑world tests reaching 370,000 while keeping CPU stable.

Push performance: Under high‑concurrency writes, the framework pushes 100‑120k updates per second externally. With 1,000 servers and 100 hot keys per worker, it delivers 100k pushes per second within one second. Pure push mode can sustain 400‑600k pushes per second, peaking at 800k for a few seconds.

Overall throughput (write + push) is stable around 700k operations per second.

In real scenarios, a 1:1000 ratio (one worker supporting 1,000 service instances) yields massive storage savings, e.g., reducing Redis cluster expansion.

Introduction

The framework detects any sudden hot requests—hot data (e.g., a product receiving a burst of requests), hot users (crawlers, bots), hot APIs—and does so with millisecond precision.

Detected hot keys are pushed to the JVM memory of all deployed machines, allowing clients to cache hot products locally, reject hot users, or circuit‑break hot APIs, while maintaining consistency across the cluster.

Core Functions

Hot data detection and push to all servers.

Applicable Scenarios

MySQL hot data local cache

Redis hot data local cache

Blacklist user local cache

Crawler user rate limiting

Interface/user‑level rate limiting

Single‑machine interface/user rate limiting

Cluster‑wide user rate limiting

Cluster‑wide interface rate limiting

Worker Performance

Every 10 seconds the system logs totalDealCount, showing 2.7‑3.1 million keys processed per interval, roughly 300k QPS.

Only a few machines are needed to handle massive real‑time detection and push tasks, far cheaper than scaling a Redis cluster.

Using protobuf serialization further improves performance; at 360k+ pushes per second CPU stays around 60%, with stress tests lasting over 5 hours without anomalies.

Interface Snapshot