Redis Hot Key Detection and Kernel-Based Real-Time Statistics

Redis hot keys refer to keys that receive a very high number of accesses within a short period, consuming excessive CPU and potentially degrading overall performance.

Common detection methods include:

Redis‑cli hotkeys parameter (scans whole keyspace using scan + object freq ).

MONITOR command with external analysis tools.

Packet capture of Redis traffic and RESP parsing.

Client/Proxy side collection and aggregation.

Each method has drawbacks such as requiring LFU eviction policy, poor real‑time capability, high overhead, or invasive client changes.

To overcome these issues, DeWu built a kernel‑level hot‑key statistics module inside Redis‑server.

Key features of the kernel‑based solution:

LRU queue (configurable size) tracks per‑second access counts; O(1) operations.

When a count exceeds a configurable threshold, the key is marked as hot and added to a hot‑key queue.

Three subscription channels (read, write, invalidation) allow clients or proxies to receive real‑time notifications.

Hot‑key logs can be queried or reset via custom commands.

Data structures (shown in code) use bit‑fields to store notification flag, access count, and timestamp:

#define HOTKEY_NOTIFIED_BIT 1
#define ACCESS_COUNT_BITS 16
#define ACCESS_TIME_BITS 46

typedef struct hotkeyRecord {
    uint64_t notified:HOTKEY_NOTIFIED_BIT;      // hotkey notification flag
    uint64_t same_period:HOTKEY_NOTIFIED_BIT;   // one notification per second
    uint64_t count:ACCESS_COUNT_BITS;           // access count
    uint64_t access_time:ACCESS_TIME_BITS;      // start time in ms
} hotkeyRecord;

Log entry structure:

#define HOTKEY_NOTIFIED_BIT 1
#define ACCESS_COUNT_BITS 16
#define LOG_TIME_BITS 46

typedef struct hotkeyLogEntry {
    uint64_t notified:HOTKEY_NOTIFIED_BIT;
    uint64_t access_count:ACCESS_COUNT_BITS; // hotkey count
    uint64_t access_time:LOG_TIME_BITS;      // start time in ms
    unsigned type;
    void *key;
} hotkeyLogEntry;

Commands for reading and resetting logs (example):

// Query read hot‑key log length
readHotkeyLog len
// Reset read hot‑key log
readHotkeyLog reset
// Get read hot‑key log (default length)
readHotkeyLog get
// Get with length or index
readHotkeyLog get [len]
readHotkeyLog get [index] [len]

The solution provides strong real‑time capability, detailed hot‑key information (time, count, type, read/write), and supports both subscription‑based alerts and log queries.

Advantages:

Real‑time statistics per second.

Rich information for each hot key.

Supports subscription channels and log queries.