Task Queues at Ten‑Million QPS: From Synchronous Processing to Queue‑Based Architecture

Why Synchronous Processing Fails at Ten‑Million QPS

During a promotion, the order‑submission latency spikes from 80 ms to 3 s because the request synchronously calls downstream services such as coupon issuance, points recording, notification push, data‑warehouse sync, and risk scoring. Each call blocks the request thread, leading to resource occupation, fault contagion, and capacity rigidity.

Resource occupation : If five downstream calls each take 50 ms, a single order consumes 250 ms of thread time. On a 4‑core machine with 200 threads, the theoretical throughput is only 800 req/s, far below the required ten‑million QPS.

Fault contagion : A slowdown or failure in any downstream service propagates up the call chain, causing the entire order flow to stall or fail, which can snowball into a system‑wide outage.

Capacity rigidity : The overall throughput is limited by the slowest downstream component, so the order service cannot be faster than, for example, the coupon service.

What Queue‑Based Architecture Changes

Queueing separates "must‑finish‑now" actions (e.g., inventory deduction, order write) from "can‑finish‑later" actions (e.g., coupon issuance, points recording, notification). Core actions remain synchronous to guarantee strong consistency, while optional actions are enqueued for asynchronous consumption, reducing the perceived order latency to the sum of core actions only.

Queueing breaks the time‑coupling between producer and consumer: the producer merely pushes a task into the queue and returns, while the consumer processes tasks at its own pace.

Three Levels of Decoupling

Traffic decoupling : The queue acts as a buffer, turning traffic spikes into a steady consumption rate, which is the essence of peak‑shaving.

Fault decoupling : If a downstream service fails, its tasks remain in the queue and are processed once the service recovers, isolating the failure to the affected task subset.

Evolution decoupling : Adding new consumers (e.g., recommendation sync, anti‑fraud checks) only requires subscribing to the existing task topic; the order service code stays unchanged.

When to Queue a Task

A task should be queued when the user does not need an immediate result and the task can tolerate eventual consistency. The article provides a classification table (omitted here) that groups typical tasks accordingly.

Do not queue tasks merely for asynchrony; if a task is fast, rarely fails, and the user expects an instant outcome, making it asynchronous adds unnecessary complexity.

Delivery Guarantees: At‑Least‑Once

Message loss can occur at three points: producer‑to‑queue, queue persistence, and queue‑to‑consumer. The article recommends waiting for acknowledgments, persisting to disk with replication, and confirming after consumer processing to achieve at‑least‑once delivery, accepting possible duplicates.

Idempotency: The Baseline Requirement

Because duplicates are inevitable, consumer logic must be idempotent. Common approaches include:

Unique‑key deduplication using a global ID table or cache.

State‑machine constraints where repeated operations leave the state unchanged.

Database unique constraints that reject duplicate writes.

In practice, a combination of fast cache deduplication, database constraints, and idempotent state transitions is used.

Backlog Management

When production outpaces consumption, messages accumulate, causing latency explosion, storage pressure, and recovery difficulty. Prevention focuses on elastic consumption via partitioned queues; each partition can be scaled independently. The article shows a diagram of partitioned consumption.

Trade‑offs include limiting parallelism per partition and managing metadata overhead. During overload, strategies such as discarding low‑priority tasks, scaling consumer clusters, or diverting backlog to a dedicated "catch‑up" cluster are recommended, together with real‑time monitoring of queue depth and latency.

Ordering vs. Parallelism

Strict global ordering forces single‑threaded consumption, which cannot sustain ten‑million QPS. The common compromise is "partition‑level ordering": tasks that require order are routed by a key (e.g., user ID) to the same partition, where they are consumed sequentially, while different partitions are processed in parallel.

Key selection must balance ordering needs against load skew; a hot key can cause partition imbalance and backlog.

Evolution from Million to Ten‑Million QPS

At 100 k QPS, many systems lack a dedicated queue and rely on in‑process thread pools, which lose tasks on restart and cannot peak‑shave. At 1 M QPS, persistent message middleware becomes standard, and the focus is on extracting all splittable synchronous work.

At 10 M QPS, the queue itself must scale: partitioning, sharding, multi‑cluster deployment, elastic consumer scaling, and rock‑solid idempotency, ordering, and backlog controls become mandatory.

Takeaway

By moving non‑critical actions into a task queue, the core order path remains fast and resilient, while the asynchronous subsystem handles delivery guarantees, idempotency, backlog, and ordering. Teams should audit their own critical flows, identify actions that can be queued, and prepare for the associated challenges.