How JD’s Hotkey Framework Detects and Pushes Hot Data in Milliseconds

Project Overview

JD App backend middleware detects hot data in milliseconds and pushes it to the memory of all server nodes, dramatically reducing the query pressure of hot keys on the data layer.

Core Functionality

Detects bursty hot requests (hot data, hot users, hot interfaces) with millisecond precision, pushes the hot keys to the JVM memory of every deployed machine, and allows clients to decide how to handle them (local cache, reject, circuit‑break, default response). Hot keys remain consistent across the cluster.

Applicable Scenarios

MySQL hot data local cache

Redis hot data local cache

Blacklist user local cache

Crawler user rate limiting

Interface‑ and user‑dimension rate limiting

Single‑machine interface/user rate limiting

Cluster‑wide user‑dimension rate limiting

Cluster‑wide interface‑dimension rate limiting

Performance Metrics

Detection: An 8‑core worker can process 160 000 key detection tasks per second; a 16‑core worker handles over 300 000, with real‑world tests reaching 370 000 without CPU overload.

Push: Under high‑concurrency writes the framework pushes about 100 000 keys per second steadily; in pure push mode it can sustain 400‑600 000 pushes per second, with a peak of 800 000.

Real‑World Validation

During JD’s 2020 618 and Double 11 promotions, the framework processed billions of hot keys daily, captured massive crawler traffic, and pushed hot items to service memory, cutting backend query load dramatically. Worker cluster throughput peaked at 15 million operations per second, and hot‑key‑based local caching accounted for over 50 % of total traffic, halving Redis load.

Architecture Design

The framework relies on Netty for communication and does not depend on custom components. Clients send keys to workers, which perform distributed aggregation and push hot keys back to clients.

Components

etcd cluster : High‑performance configuration center storing rules, worker IPs, detected hot keys, and manually added keys.

Client jar : Integrated into services to report keys, listen to etcd changes, and maintain a local Caffeine cache of hot keys.

Worker cluster : Independent Java processes that connect to etcd, report their IPs, aggregate key counts, and push hot keys when thresholds are met.

Dashboard console : Visual Java application connected to etcd for setting key rules, monitoring hot keys, and manually managing them.