Ten‑Million QPS Architecture: From Strong to Eventual Consistency with Layered Design

Why Strong Consistency Is Expensive

Strong consistency means every read after a successful write sees the latest value, which requires coordination among all replicas. The coordination cost appears as two‑phase commit with at least two network round‑trips, locking resources while waiting for acknowledgments. This also sacrifices availability per the CAP theorem: if a participant does not respond, the system must either wait indefinitely or abort the write. Moreover, tail latency is amplified because the overall transaction latency equals the slowest participant; a single 500 ms pause in a five‑service chain makes the whole request 500 ms, which is tolerable at million QPS but catastrophic at ten‑million QPS.

Strong consistency performs all confirmations within the user’s waiting time, whereas eventual consistency pushes alignment after the user leaves, guaranteeing convergence only when no new writes occur.

Eventual Consistency Is Not Inconsistency

Eventual consistency is a formally defined model: if no new writes happen, all reads will eventually return the same latest value within a bounded, observable window (typically milliseconds to seconds). It guarantees three properties: no data loss, convergence, and a bounded convergence window.

Whether the short inconsistency window is acceptable depends on the business impact of reading stale data. Social feeds, order status visible to sellers, and like counts can tolerate a few‑second lag because the stale view does not cause incorrect decisions. In contrast, bank account debits, seat or inventory reservations cannot tolerate stale reads, as they may lead to overdraft or overselling.

Consistency Is Layered by Data

Consistency should be applied per data field rather than globally. In the payment example, only account balance deduction and the transaction record require strong consistency; order status, inventory decrement, user growth points, and merchant notifications can be eventually consistent.

The guiding principle: keep strong consistency for operations that affect irreversible, limited resources (money, inventory); use eventual consistency for derived data that can tolerate delayed alignment.

Ensuring “Eventually Consistent” Works

Asynchronous alignment introduces the risk of lost or failed messages. The classic “dual‑write inconsistency” occurs when the database write succeeds but the message fails, or vice‑versa. The industry‑standard solution is the local message table (transactional outbox), where the message is stored in the same local transaction as the core data, guaranteeing atomicity.

An independent dispatcher scans the outbox for undelivered messages, retries delivery, and marks them as completed. Because delivery is “at‑least‑once,” downstream processing must be idempotent, turning “at‑least‑once” into “effectively once.”

Compensation and Reconciliation

If a downstream step fails after some earlier steps succeeded (e.g., balance deducted but user growth update fails), compensation (Saga) rolls back the effects with inverse operations (e.g., refund). Compensation differs from database rollback: it leaves a compensating record rather than erasing history.

Reconciliation runs periodically to compare data across systems, catching rare mismatches that escaped the outbox or compensation mechanisms. It serves as the final safety net, especially in financial‑grade systems where daily reconciliation is standard.

Handling the User‑Facing Inconsistency Window

To avoid user confusion during the short window, systems can enforce “read‑your‑write” by forcing a read from the primary for the same user, or optimistically display the submitted content on the front‑end. Additionally, using intermediate status values (e.g., “Payment Confirming”) sets correct expectations. Finally, the convergence window must be observable and alertable; monitoring latency and setting upper bounds prevents the window from silently expanding.

Evolution of Consistency Strategies with Scale

At 100 k QPS, a monolithic strong‑consistent design is simplest. At 1 M QPS, services are split, and clearly asynchronous derived data (points, notifications, search indexes) are moved to eventual consistency, keeping the core transaction strong. At 10 M QPS, consistency must be fine‑grained to the field level: amount fields stay strong, status fields become eventual, and statistics may be only approximately consistent. Infrastructure must deliver sub‑millisecond outbox latency, near‑real‑time reconciliation, and automated compensation.

The overarching lesson: only pay the strong‑consistency cost for the truly critical data; let the rest converge over time, thereby unlocking massive throughput at ten‑million QPS scale.