Lock Optimization at Ten‑Million QPS: From Coarse Locks to Lock‑Free Designs

During a promotional traffic spike the QPS rose from 2 M to nearly 8 M, P99 latency jumped from 5 ms to 800 ms while CPU stayed at 60 %. Flame graphs showed most CPU time spent in __lll_lock_wait and many futex and context‑switch calls, indicating that threads were queuing on a global lock.

The root cause is a global counter protected by a single lock. At the million‑QPS level the lock is barely sufficient, but at ten‑million QPS it serialises all work, turning parallel cores into a single‑threaded bottleneck. Using Amdahl’s law, even 5 % of work locked serially caps speed‑up around 20×, far below the 10× throughput needed.

Lock contention incurs three major costs: (1) system‑call and thread‑blocking overhead via futex, which can happen millions of times per second; (2) full context switches that flush pipelines and pollute caches; (3) false sharing, where unrelated variables share a cache line, causing cache‑coherence traffic.

The first evolution step is to replace the single coarse lock with many fine‑grained locks. In JDK 7, ConcurrentHashMap used segment locks (default 16 segments); in JDK 8 it switched to bucket‑level locks, using CAS for empty buckets and synchronized only when a bucket contains a list or tree, raising concurrency from 1 to hundreds of buckets. The trade‑off is higher memory usage and management complexity.

The next step separates read‑heavy workloads from writes. A ReadWriteLock allows concurrent reads but can starve writers. Java 8 introduced StampedLock with an optimistic read mode: a thread reads a version stamp, accesses data without locking, then validates the stamp; if it changed, the read falls back to a normal lock.

The JVM already performs several automatic lock optimisations. Lock elimination removes locks on objects that never escape a thread (e.g., a locally created StringBuffer). Lock coarsening merges many small locks inside a loop into a single larger lock to avoid repeated acquire/release. Lock inflation upgrades a lightweight lock to a heavyweight one only when contention is detected, using spinning and adaptive spinning before falling back to kernel suspension.

The third evolution adopts lock‑free techniques based on CAS ( Compare‑And‑Swap). Java’s AtomicInteger and AtomicLong implement a spin‑retry loop: read the current value, compute a new one, attempt CAS, and repeat on failure. High contention can cause many retries, degrading throughput. The ABA problem is illustrated, and AtomicStampedReference solves it by attaching a version number to the value.

For a global counter at ten‑million QPS, a plain AtomicLong still suffers cache‑line ping‑pong. Replacing it with LongAdder spreads updates across many cells, each padded to avoid false sharing, dramatically reducing contention. After the switch, CPU utilisation rose to 100 % and P99 latency fell back to single‑digit milliseconds.

Beyond LongAdder, fully lock‑free data structures such as ConcurrentLinkedQueue (CAS‑driven queue) and the Disruptor’s ring buffer achieve several‑fold higher throughput by eliminating locks and padding critical fields. However, any remaining shared state still incurs cache‑coherence traffic.

To eradicate sharing entirely, one can use ThreadLocal /per‑CPU data (the idea behind LongAdder) or Linux’s RCU pattern, where reads are lock‑free and writes copy‑modify‑replace the data structure. Correct memory ordering with volatile, memory barriers, and the happens‑before rule is essential; otherwise lock‑free code may exhibit subtle, architecture‑specific bugs.

The overall evolution is a consistent theme: shrink the contention scope—from a single global hotspot, to segmented locks, to paying only when conflicts occur, and finally to eliminating shared state. The choice of step depends on scale: coarse locks suffice up to ~100 k QPS, fine‑grained locks and read‑write separation become necessary around 1 M QPS, and lock‑free structures or per‑CPU data are required to sustain ten‑million QPS.