Four Levels of Concurrency Control: Optimistic Locks to Message Queues for Million‑QPS

Why concurrency updates are hard

In high‑traffic systems the biggest challenge is not receiving requests but ensuring that a shared business state is updated correctly under massive parallelism. Typical operations such as inventory deduction, balance change, coupon redemption, or order status transition all compete for the same resource, and a naïve design can lead to overselling, duplicate charges, database overload, message backlog, or uncontrolled compensation tasks.

Four dimensions of conflict

Data conflict – multiple requests modify the same row.

Resource conflict – connection pools, thread pools, Redis or MQ partitions become bottlenecks.

Status conflict – different subsystems have inconsistent views of the same business fact.

Temporal conflict – request order, message arrival order, retry order, and compensation order may diverge from business expectations.

If these dimensions are not separated, teams tend to add a lock locally and end up with a system that is both slow and fragile.

Four progressive levels of concurrency control

1. Optimistic lock (CAS in the database)

Assumes conflicts are rare. A version column is added to the table and the update checks that the version has not changed.

if current_version == expected_version:
    update data and version = version + 1
else:
    fail

Advantages: non‑blocking, high concurrency. Drawbacks: when conflict rate rises, retry storms waste CPU and DB resources. The article lists three common pitfalls: treating retry cost as free, applying optimistic lock on hotspot rows, and ignoring business‑level idempotency.

Typical use cases: read‑heavy, write‑light workloads, user‑profile updates, non‑hot inventory, background configuration changes.

2. Pessimistic lock (row‑level SELECT … FOR UPDATE )

Assumes conflicts are frequent, so the row is locked for the whole transaction. SELECT ... FOR UPDATE Pros: conflicts are serialized, strong consistency is easy to guarantee. Cons: lock wait increases latency, long transactions block subsequent requests, connection pools can be exhausted. Common engineering mistakes include holding the lock while calling remote services, performing cache reads, or doing complex object assembly inside the transaction.

Typical scenarios: account balance deduction, fund transfer, quota reservation where “must not be wrong” outweighs throughput concerns.

3. Redis distributed lock

Extends mutual exclusion across multiple application instances. The minimal correct implementation uses SET NX PX to acquire the lock and a Lua script to release it safely.

SET lock:resource uniqueToken NX PX 30000

if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
else
    return 0
end

Redis lock is the first gate, not the final truth. The database still enforces the ultimate constraints. Risks include lock renewal failures, master‑slave switch‑over loss, and turning Redis into a global bottleneck if over‑used.

Typical use cases: cross‑instance mutual exclusion, protecting idempotent entry points, hotspot mutex.

4. Message‑queue serialization

When traffic spikes, the real problem becomes “why are millions of requests trying to update the same row at the same time?”. The solution is to front‑load validation, then enqueue the request. Consumers process messages in order per resource (e.g., productId as the Kafka key) and perform the final database update.

ProducerRecord<String, StockCommand> record = new ProducerRecord<>(
    "stock-command-topic",
    String.valueOf(command.productId()),
    command);

Consumer example (simplified):

@KafkaListener(topics = "stock-command-topic", groupId = "stock-service")
public void onMessage(ConsumerRecord<String, StockCommand> record, Acknowledgment ack) {
    StockCommand cmd = record.value();
    int inserted = commandLogMapper.insertIgnore(cmd.commandId(), cmd.orderId());
    if (inserted == 0) { ack.acknowledge(); return; }
    try {
        ProductStockDO stock = stockMapper.findByProductId(cmd.productId());
        int updated = stockMapper.reserveWithOptimisticLock(
            cmd.productId(), cmd.quantity(), stock.version());
        if (updated == 0) throw new BizException("stock shortage or version conflict");
        commandLogMapper.updateStatus(cmd.commandId(), "SUCCESS");
        ack.acknowledge();
    } catch (Exception e) {
        commandLogMapper.updateStatus(cmd.commandId(), "FAILED");
        throw e;
    }
}

Key points: idempotent log table prevents duplicate consumption; the final DB update still uses optimistic locking to guarantee correctness.

When to upgrade to the next level

If conflict rate exceeds the DB’s comfortable zone, consider moving from optimistic to pessimistic lock.

If conflicts are high and strong consistency is mandatory, use pessimistic lock but keep transactions short.

If the contention is across multiple instances, introduce Redis lock before DB lock.

If peak traffic overwhelms any lock‑based solution, switch to the MQ‑based asynchronous pipeline.

Composite architecture for million‑QPS

Gateway & edge rate limiting : CDN cache, IP/device/user limits, signature & captcha checks.

Redis eligibility & idempotency : atomic token bucket, duplicate‑submission guard, activity‑window check.

MQ peak‑shaving & ordered consumption : route messages by productId to the same partition, scale consumer groups horizontally.

Database final confirmation : conditional updates, unique constraints, state‑machine transitions.

In hotspot scenarios, further split inventory into shards (e.g., product_1001 → shard_0, shard_1 …) to increase parallelism, but only after the layered design is proven.

Engineering capabilities beyond code

Idempotency design : request‑level ( requestId), message‑level ( commandId), and state‑level guarantees.

Timeout & compensation : scheduled tasks, dead‑letter queues, manual reconciliation.

Multi‑layer rate limiting : gateway, application thread‑pool, resource‑level (per‑product) limits.

Observability : metrics for optimistic‑lock retries, lock wait time, Redis‑lock miss rate, Kafka lag, stock‑pre‑reserve vs release delta, dead‑letter size.

Reconciliation & audit : separate logs for orders, inventory movements, and compensation actions.

Common production pitfalls

Assuming a lock guarantees overall consistency – it does not prevent duplicate requests or out‑of‑order state transitions.

Believing Redis lock is a superior replacement for DB lock – they solve different problems.

Thinking MQ solves all issues – it only shifts peak load; idempotency, state convergence, and compensation remain necessary.

Ignoring failure paths – retry storms, duplicate callbacks, and master‑slave switch failures are often the real breakers.

Over‑engineering too early – introduce the simplest viable solution first and keep upgrade paths.

Practical checklist before launch

Define unique resource keys and lock granularity.

Implement API‑level idempotency.

Limit retry attempts and add random back‑off.

Retain final DB constraints (unique indexes, conditional updates).

Model inventory as pre‑reserve → confirm → release states.

Ensure MQ consumer idempotency and dead‑letter handling.

Monitor core metrics (optimistic‑lock retries, lock wait, Redis‑lock failures, Kafka lag, stock‑reserve vs release delta).

Provide periodic reconciliation jobs.

Stress‑test hotspot items, duplicate requests, message backlog, and instance restart scenarios.

Validate Redis, MQ, and DB behavior during failover.

Only when most of these items are satisfied should a system claim to be ready for “high‑concurrency production”.