How to Solve Distributed Cache Consistency Issues with Lazy Updates

Cache Aside Pattern

The classic cache‑plus‑database read/write model is the Cache Aside Pattern: on read, check the cache first; if missing, read from the database, store the result in the cache, and return it. On update, write to the database then delete the cache.

Why delete the cache instead of updating it? In complex scenarios the cached value may be derived from multiple tables, requiring expensive calculations. Updating such a cache on every write can be costly, especially when the cache is rarely accessed. Deleting the cache defers the expensive computation until the next read, following a lazy‑calculation approach similar to lazy loading in ORM frameworks.

Basic Cache Inconsistency Problem and Solution

Problem: update the database then delete the cache. If cache deletion fails, the cache holds stale data, causing inconsistency.

Solution: delete the cache first, then update the database. If the database update fails, the cache remains empty, so reads fall back to the old database value and refresh the cache correctly.

Analysis of More Complex Inconsistency Scenarios

When data changes, the cache is deleted before the database update completes. A concurrent read may find the cache empty, fetch the old database value, and repopulate the cache, while the pending write later updates the database, leaving the cache and database out of sync.

This can happen under high‑traffic, high‑concurrency workloads where many reads and writes target the same data.

Proposed Queue‑Based Lazy Update Solution

When updating data, route the operation (identified by a unique key) to an internal JVM queue. On read, if the cache is missing, enqueue a read‑plus‑cache‑refresh task for the same key.

Each queue is serviced by a single worker thread that processes operations sequentially. This ensures that a write deletes the cache, then the subsequent read sees an empty cache, enqueues a refresh, and the worker updates the cache after the database write completes.

Optimization: filter duplicate cache‑update requests in the queue; if a refresh is already pending, skip adding another.

When the worker finishes the previous database write, it reads the latest value and writes it to the cache. If a read request times out while waiting, it can fall back to reading the current (possibly stale) database value.

High‑Concurrency Considerations

1. Read request blocking – Because reads may wait for the queue to process updates, set appropriate read‑timeout limits and test under realistic load to ensure acceptable latency.

2. Read request surge – Simulate traffic spikes to verify that the system can handle many concurrent reads without excessive queuing.

3. Request routing across multiple service instances – Ensure that both write and cache‑update operations for the same key are routed to the same instance (e.g., using Nginx hash routing) to maintain ordering.

4. Hot‑item routing imbalance – Distribute hot keys across multiple queues or instances to avoid overloading a single machine.

Overall, the solution works well when write frequency is low relative to read traffic, which is typical for many cache‑backed systems.