How JD’s Hotkey Framework Detects and Mitigates Hot Data in Milliseconds

Project Introduction

JD App backend middleware detects hotspot data at millisecond granularity and pushes it to server‑cluster memory, greatly reducing pressure on hot keys for the data layer.

It precisely captures any sudden hotspot requests—such as massive requests for a single product, hot users (crawlers, bots), or hot interfaces—and pushes these hot keys to the JVM memory of every deployed machine, allowing clients to cache hot items, reject hot users, or circuit‑break hot interfaces, while maintaining consistency across the cluster.

Core Functions

Hot data detection and push to each server.

Applicable Scenarios

MySQL hot data local cache

Redis hot data local cache

Blacklist user local cache

Crawler user throttling

Interface and user‑level rate limiting

Single‑machine interface/user throttling

Cluster‑wide user throttling

Cluster‑wide interface throttling

Performance Metrics

Detection performance: on an 8‑core single machine, the worker can handle 160,000 key detection tasks per second; a 16‑core machine processes over 300,000 per second, with real‑world tests reaching 370,000 without CPU issues.

Push performance: under high‑concurrency writes, the system pushes steadily at 100,000–120,000 pushes per second; with 1,000 servers, 100 hot keys per second result in 100,000 pushes per second, all delivered within one second. In push‑heavy scenarios, the framework sustains 400,000–600,000 pushes per second, peaking at 800,000 for a few seconds.

Real‑World Validation

During JD’s 2020 618 and Double‑11 promotions, the framework processed billions of hot keys daily, accurately capturing crawlers, bots, and popular products, and pushing them to all service memories within milliseconds, cutting data‑layer query pressure and boosting application performance.

During peak events, the hotkey worker cluster achieved a throughput of 15 million keys per second, and hot keys accounted for over 50% of total traffic, shifting most requests to local cache and relieving more than half of the Redis load.

Architecture Design

The framework is lightweight, with no custom components and no dependency on Redis; it relies on Netty connections. Clients send keys to be tested, workers compute distributed hot‑key statistics, and hot keys are pushed back to clients.

The system consists of four parts:

etcd cluster : a high‑performance configuration center storing rules, worker IPs, detected hot keys, and manually added hot keys.

Client‑side JAR : integrated into services to check if a key is hot, report keys, listen for rule changes, worker updates, and hot‑key updates, and cache hot keys locally using Caffeine.

Worker cluster : independent Java processes that connect to etcd, report their IPs, receive keys from clients, aggregate counts, and push hot keys to clients when thresholds are met.

Dashboard console : a visual Java application that also connects to etcd, allows setting key rules (e.g., 20 occurrences within 2 seconds), records detected hot keys, and enables manual addition or removal of hot keys for clients.

Project URL

https://gitee.com/jd-platform-opensource/hotkey