JD-hotkey: Real-time Hot Key Detection to Offload Redis in Flash Sales

Introduction: The Hot Key Problem in Flash Sales

A flash sale product detail page normally handles a few hundred QPS, but at event start traffic spikes to tens of thousands. All requests hit the same Redis key (the product ID), saturating Redis CPU and blocking Tomcat threads waiting for Redis responses. Caching solves repeated queries but cannot handle burst traffic concentrated on a single key. The core issue is not caching but detection: you must know which key becomes hot before it overwhelms Redis.

1. What Makes a Key "Hot"?

Hot keys are bursty, not predictable. An ordinary product can become a bestseller in seconds due to a livestream mention or platform push. Pre-caching all millions of keys is impossible. JD's data shows that during major promotions, hot keys detected by hotkey account for over 50% of total application access, shifting most requests to local lookups and cutting Redis load by more than half.

2. How JD-hotkey Detects Hot Keys

JD-hotkey uses an independent detection cluster (Worker) plus a client JAR, with zero dependency on Redis internals. The flow has four steps:

Client collection : Before querying Redis, business code calls JdHotkey.isHotKey(key). The client batches keys locally and reports them to Workers every 500 ms.

Worker aggregation : Workers receive keys from all clients and count occurrences using a sliding window (e.g., 20 occurrences in 2 seconds = hot).

Push to clients : Once a key is judged hot, the Worker pushes it via long-lived connections to all subscribed clients. Clients store the key in a local Caffeine cache.

Business decides usage : The framework only signals "this key is hot". On isHotKey returning true, the application checks local cache via get(key); if missing, it fetches from Redis once and calls smartSet(key, value) to populate the local cache. Subsequent requests read from JVM memory with zero network overhead.

This decoupling of detection and caching gives flexibility: the business can choose local caching, rate limiting, or circuit breaking.

3. Code Integration Walkthrough

3.1 Maven Dependency

<dependency>
<groupId>com.jd.platform</groupId>
<artifactId>hotkey-client</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>

3.2 Configuration (application.yml)

hotkey:
  app-name: your-app-name
  etcd-server: http://127.0.0.1:2379
  caffeine-size: 50000
  push-period: 500

3.3 Business Logic Example

@Service
public class ProductService {
    @Autowired
    private JdHotkey jdHotkey;
    @Autowired
    private RedisTemplate<String, Object> redisTemplate;

    public Product getProduct(String productId) {
        String key = "product_" + productId;
        if (jdHotkey.isHotKey(key)) {
            Object cached = jdHotkey.get(key);
            if (cached != null) {
                return (Product) cached;
            }
            Product product = (Product) redisTemplate.opsForValue().get(key);
            if (product != null) {
                jdHotkey.smartSet(key, product);
            }
            return product;
        }
        return (Product) redisTemplate.opsForValue().get(key);
    }
}

Logic breakdown: isHotKey returns false: key not hot, normal Redis path. The call still reports the key to Worker for counting. isHotKey returns true: Worker has marked it hot. Check local Caffeine first (zero network). On miss, fetch from Redis once, then smartSet to cache locally. smartSet only writes for hot keys; non-hot keys are ignored.

One code path handles both cold and hot states automatically.

4. Performance Benchmarks

Logs show 2.7–3.1 million keys processed per 10 seconds (~300k QPS). With protobuf serialization, CPU stays around 60% at 360k+ QPS, stable for over 5 hours; at 300k QPS, stable for days. A single Worker supports 1,000 business servers, costing far less than scaling Redis clusters.

5. Pros and Cons

Pros

Lightweight, no Redis coupling : No Redis source changes, no client dependency.

Detection-cache decoupling : Framework only signals heat; business chooses action.

Cluster consistency : Hot keys pushed to all application JVMs, avoiding split-brain.

Millisecond latency : 500 ms reporting interval, push delay in milliseconds.

Low cost : 1 Worker ≈ 1,000 servers.

Battle-tested : Used in JD 618 and Double 11 since 2020, detecting billions of keys daily and identifying crawlers.

Cons

Extra infrastructure : Requires Etcd cluster, Worker cluster, Dashboard.

Code intrusion : Must explicitly call isHotKey at cache-read points.

No value storage : Values live in local Caffeine; eviction means fallback to Redis.

Rule tuning needed : Thresholds (e.g., 2 sec/20 hits) must match business; too sensitive wastes memory, too slow misses the spike.

6. Applicable Scenarios

Flash sale / big-promo product detail — ✅✅✅ Strongly recommended. Reason: Burst traffic on single key; local cache absorbs load.

Crawler / scraper detection — ✅✅✅ Strongly recommended. Reason: isHotKey on user ID enables instant rate limiting.

Hot API circuit breaking — ✅✅✅ Strongly recommended. Reason: Push hot interface key, apply local circuit breaker.

Redis hot key local caching — ✅✅✅ Strongly recommended. Reason: Classic use case, cuts Redis read pressure.

Blacklist user local caching — ✅✅ Recommended. Reason: High-frequency blocked users cached locally for fast reject.

Low-traffic internal systems — ❌ Not recommended. Reason: Etcd + Worker ops cost not justified.

No burst traffic — ⚠️ Evaluate. Reason: Hot key problem not prominent, limited gain.

7. Conclusion

JD-hotkey shifts hot key handling from "post-mortem monitoring" to "pre-emptive in-memory push". The core contradiction is not whether to cache, but that you don't know which key will turn hot next second. A lightweight Worker cluster performs distributed counting, judges heat in milliseconds, and fans out to every application JVM. Business code adds a single isHotKey check; the framework handles the rest. You must deploy Etcd and Workers, instrument your code, and tune thresholds. But for flash sales, major promotions, and crawler detection, the trade-off yields over 50% Redis pressure reduction at negligible cost.

References

Gitee repository : https://gitee.com/jd-platform-opensource/hotkey

JD Cloud developer article : https://developer.jdcloud.com/article/2854

Hot key detection design & practice : https://developer.aliyun.com/article/1655928