How to Build a Low‑Latency Timeout Center with Redis: Architecture & Design

Background

Many products require lifecycle‑related designs where actions must be taken when specific time nodes are reached. The TimeOutCenter (TOC) stores and schedules timeout tasks; when a task's timeout expires, it must be dispatched immediately, especially for low‑latency scenarios.

Traditional High‑Latency Scheme

The classic TOC stores tasks in a timeout task database. A timer triggers a database scanner that loads expired tasks into an in‑memory queue for business processors. This design relies on sharding (e.g., 8 databases, 1024 tables) to handle massive task volumes, but suffers from several drawbacks:

The timer interval adds latency.

Each table can be processed by only one machine, limiting concurrency.

Scanning large tables is time‑consuming.

Overall latency equals timer interval plus scan time, which can be large under heavy load.

Low‑Latency Scheme

The new design still stores tasks in the same database schema, but introduces a Redis‑based timing system. After a task is written to the database, its jobId and actionTime are also placed into a Redis cluster. When the timeout expires, the jobId is popped from Redis, the full task is fetched from the database, processed, and its status is updated.

Redis Storage Design

Tasks are organized into Topics , each with a configurable number of slots (a power of two). Messages are stored in Redis Sorted Set s, using the timestamp as the score. Slots are addressed by keys like #{topic}_#{index}.

StoreQueue

Each slot holds a Sorted Set where the score is the execution timestamp; popping the smallest score retrieves the next due message.

PrepareQueue

To guarantee at‑least‑once delivery, consumers first move a message from StoreQueue to PrepareQueue via a Lua script, process it, then either delete it (on success) or move it back (on failure). The score in PrepareQueue is timestamp*1000 + retryCount, enabling timeout‑based rollbacks.

DeadQueue

After 16 failed retries, a message is moved to DeadQueue for manual inspection.

Message Production

Producers calculate the appropriate slot key (using CRC32 of a provided slotBasis or the message content) and add the message to the Redis Sorted Set with actionTime as the score.

Message Consumption

Multiple workers (Woker) run on each machine. Zookeeper assigns slots to workers so that each slot is consumed by only one worker. Workers continuously query their assigned slots with ZRANGEBYSCORE to pop messages whose score is less than the current timestamp.

At‑Least‑Once Guarantee

The two‑phase commit‑like flow ensures that a message is either successfully processed and removed or rolled back to the store after a timeout, providing reliable delivery.

Retry Control

Retry count is encoded in the score of PrepareQueue. Each failure reduces the remaining retries; when retries are exhausted, the message moves to DeadQueue.

Advantages

Low latency: direct pop from Redis eliminates database scans.

Controllable concurrency: determined by slot count and worker pool size.

High performance: Redis can handle >100k QPS with O(1) retrieval.

High availability: at‑least‑once processing and bounded retries prevent blockage.