Designing High‑Concurrency Order Processing with Distributed Locks and Redis

Click Follow Public Account

A simple order flow includes product validation, pricing, stock deduction, and order persistence, with stock deduction being the most challenging concurrent step.

To solve the concurrency problem, locking is introduced to ensure atomicity, visibility, and ordering, using either JVM locks or distributed locks.

Lock granularity should be as narrow as possible, typically at the SKU level, and lock duration must be carefully managed; Redisson's watchdog can extend lock lifetimes but may increase response time.

Combining pessimistic and optimistic locks (e.g., using Google Chubby) can provide a fallback when locks expire without sacrificing latency.

Redis offers high performance and atomic operations, making it a suitable alternative to traditional locks for stock deduction.

Order Process

Stock deduction is the bottleneck for order concurrency; using Redis's atomicity ensures data consistency while boosting performance.

2PC

Based on the two‑phase commit idea, the order is first inserted with an INIT status.

Hot‑Cold Routing

Cold‑selling items are processed via MySQL, while hot items rely on Redis for high concurrency. Item popularity is determined using a bitmap maintained by a scheduled job.

Failure Handling

When executing the Lua script for stock deduction, request timeouts can be caught and retried; Redis outages require more complex handling.

Degradation

If Redis fails, the system falls back to the cold‑item flow, but must ensure MySQL is not overwhelmed and trigger circuit‑breaker and rate‑limiting mechanisms.

Post‑deduction actions include inserting inventory logs, updating order status, sending coupon redemption messages, and logging.

Inventory logs record the binding between orders and stock, useful for rebuilding cache.

Coupon redemption is asynchronous; a local message table and a retry job ensure idempotency.

JVM crashes must be handled; abnormal orders need stock rollback via a scheduled task.

JOB2

If Redis lacks inventory logs, the deducted stock cannot be verified.

Both Redis and JVM crashes produce invalid orders that must have their stock returned.

In extreme cases where Redis crashes after a Lua script executes, the atomicity guarantee is lost, and the missing stock must be handled, possibly via Redis HA.

Rebuild Stock Cache of SKU

Remaining stock = (total stock – reserved stock) – (deduplicated MySQL and Redis logs).

The stock service uses an a/b model to separate total and reserved stock, reducing lock contention under high concurrency.

When Redis starts empty, stock = a – b; over time, pending messages cause a temporary discrepancy that is resolved by deduplication.

Payment Process

Only after payment is the stock‑deduction message sent to the inventory service; adding stock is straightforward, while reducing stock must avoid overselling.

Additional Remarks

Introducing an intermediate layer (e.g., Redis) improves concurrency but adds complexity; scaling Redis horizontally and handling hot‑cold routing are essential for billion‑scale SKU catalogs.

Cache sharding, bitmap‑based hotness detection, and distributed lock strategies help balance performance and consistency.

Message back‑pressure can be mitigated with producer‑consumer patterns and batch commits.

Answering Common Questions

Why does removing the distributed lock make the flow seem more complex?

Adding a distributed lock around the entire order flow actually simplifies reasoning; the core deduction logic remains unchanged.

Why does the process become complex?

Additional scheduled tasks and cache‑rebuild interfaces ensure data consistency.

Hot‑cold routing improves performance but adds routing logic.

Multiple data sources (MySQL, Redis) require consistency management.

Summary

Scenario 1: Low concurrency, MySQL sufficient

Use MySQL with an outer distributed lock, careful lock granularity, and possibly Sentinel for rate limiting.

Scenario 2: High concurrency, MySQL bottleneck

Prefer Redis for stock deduction; MySQL handles cold items or overflow.

Scenario 3: Billions of SKUs

Store hot‑item stock in Redis and cold‑item stock in MySQL; consider Redis Cluster and sharding to manage memory.

Overall, the design balances concurrency, consistency, and fault tolerance through layered architecture and appropriate use of Redis, distributed locks, and routing strategies.