Skip to content
Redis Cache Consistency: Cache-Aside, Delayed Double Delete, Binlog Fallback & Versioning

Redis Cache Consistency: Cache-Aside, Delayed Double Delete, Binlog Fallback & Versioning

This article analyzes Redis cache consistency through three real incidents, compares four read-write patterns, explains why update-DB-then-delete-cache (Cache-Aside) is optimal, details delayed double delete, binlog subscription fallback, versioning, monitoring, and a decision tree for choosing consistency levels.

Architecture Pony
Architecture Pony
Architecture Pony

Introduction: Three Real Incidents

The article opens with three production incidents that frame the cache consistency problem:

Concurrent cache updates race : Two price-update requests write to DB (79 then 89) but cache SETs reverse order, leaving stale 79 in cache. Root causes: write-cache-before-DB, forget to write DB, concurrent cache write reordering.

Cache delete fails silently : Inventory service updates DB but cache delete fails. Retry logic sends deletes to a local thread pool with no consumer and message loss. Stale cache persists for three days, causing 217 oversold items. Key lesson: failed deletes must have a visible destination (retry, dead-letter, alert).

Delayed double delete blocks business threads : Team implements delayed double delete (write DB, delete cache, sleep 500ms, delete again) but places Thread.sleep(500) in the request thread. P99 latency jumps from 40ms to 550ms; thread pool saturates under load. The pattern is valid, but the delay must run asynchronously.

All three incidents illustrate the core conflict: cache and DB are independent stores with no atomic coordination, yet business demands immediate consistency.

Technical Analysis: Four Read-Write Combinations

1.1 The Four Quadrants

Write path has two actions: update DB and operate on cache (update or delete). Four combinations:

Update cache then update DB : If DB fails, cache holds dirty data with no rollback — excluded.

Update DB then update cache : Seems reasonable but has two fatal flaws: (a) concurrent write reordering (thread A writes 79, thread B writes 89; B's cache SET finishes first, then A's overwrites with 79), and (b) write amplification on write-heavy/read-light keys (100 writes/sec, 1 read/sec — 99 wasted cache computations).

Delete cache then update DB : Window between delete and DB commit allows a read to re-populate stale value.

Update DB then delete cache (Cache-Aside) : Sole survivor; default in all major cache frameworks.

Cache-Aside still has a theoretical race: read misses cache, reads DB (old value), write commits and deletes cache, read re-populates old value. Requires three strict conditions simultaneously — probability extremely low, and TTL provides a safety net. The principle: choose the option with the smallest, most manageable failure surface.

1.2 Why Delete Instead of Update — Three Reasons

Concurrent write reordering is unsolvable for updates : Two threads updating the same cache key have no ordering guarantee relative to DB writes (unless distributed lock, cost outweighs benefit). Delete is idempotent and order-insensitive: delete twice = delete once; any order forces next read to re-fetch.

Avoid wasted computation : Cache values often require aggregation, serialization, or secondary queries. Write-heavy/read-light keys waste CPU recomputing on every write. Delete defers computation to the moment a read actually arrives (lazy evaluation).

Aggregated cache values span multiple tables : Product detail cache merges product, price, stock tables. Any source table change should invalidate the cache. Update would require the writer to recompute the full aggregate; delete pushes recomputation to the next read, zero burden on writer.

Boundary: Update cache can be justified when cache computation is extremely expensive (e.g., complex leaderboards) and read/write ratio approaches 1:1. Decision metric: probability of being read within 60 seconds after write. High probability → update; low probability → delete. Most business caches fall in the latter.

1.3 Inconsistency Window in Cache-Aside

Cache-Aside provides eventual consistency with a bounded window . Normal window = milliseconds between DB commit and cache delete completion, plus a tiny-probability stale-repopulation window (covered by TTL). Three factors actually widen the window:

Cache delete fails — window extends indefinitely until TTL expiry or next write.

Stale repopulation — window = TTL.

Master-slave replication lag — read-from-replica may return stale data.

Goal is not to eliminate inconsistency (requires distributed transactions) but to bound each window acceptably. Formula: business tolerance for stale data determines which tier of solution you need . Minutes tolerance → Cache-Aside + short TTL. Seconds tolerance → add binlog fallback. Milliseconds/strong consistency → don't use cache, read DB directly or serialize with distributed locks. Incident three's team used 500ms delayed double delete chasing near-strong consistency but paid 500ms per write — tier mismatch is the top cause of solution failure.

1.4 Delayed Double Delete: What Problem It Solves

Targets delete-cache-then-update-DB (option 3) or read-replica stale read scenarios. After DB commit, replica replication lag (tens of ms) may cause a read to fetch old value and repopulate cache. Second delete (delayed N ms) clears that stale repopulation. Three key points:

Delay must exceed master-slave sync lag + one read-round-trip ; typically 500ms–1s, observe and tune.

Second delete must be asynchronous (thread pool or message queue), never sleep in business thread — incident three's lesson.

Only fixes stale repopulation; does not fix delete failures — those still need retry mechanism. If you already use update-DB-then-delete-cache and have no read-replica stale-read issue, delayed double delete is unnecessary.

1.5 Binlog Subscription: Decouple Delete from Business Code

Business code doing update DB + delete cache has three inherent flaws: code intrusion (every write point must remember delete), hard failure handling (retry where? inside transaction can't retry, outside may lose), and missed deletes (new write point forgets delete).

Subscribing to binlog (Canal/Debezium) is the standard fix : Canal poses as MySQL replica, pulls binlog, parses row changes, publishes to MQ, consumer deletes corresponding cache keys by rule. Benefits chain: zero business intrusion, binlog events are post-commit (never delete early), MQ retry and dead-letter queue catch delete failures. Cost: new component chain and second-level latency, so it serves as fallback layer for Cache-Aside, not sole mechanism.

Incident two's fix: move delete to RocketMQ, 16 retries, then dead-letter + alert. Failed delete must have a named destination: either retry succeeds, or dead-letter is visible, never allowed to vanish.

1.6 Strong Consistency: Read/Write-Through and Distributed Locks

If business truly demands "every read sees latest write", stop patching Cache-Aside. Two paths:

Write-Through : App writes only to cache; cache layer synchronously writes DB before acknowledging. Reads and writes both go through cache layer, serializing all operations — naturally consistent. But cache layer must be custom-built (Spring Cache only reaches annotation level), and write latency doubles.

Read/Write locks / distributed lock serialization : All reads and writes for a key acquire the same lock. Strongest consistency, throughput drops to single-digit QPS.

Most "strong consistency" demands, when probed, reduce to "no stale data older than 1 second" — satisfied by Cache-Aside + binlog fallback + 1s TTL at an order-of-magnitude lower cost. Calibrate requirement first, then choose solution.

Core Flows: Complete Timeline of a Price Change

2.1 Normal Path: Update DB Then Delete Cache

Example: operator changes price 99 → 88.

T1 Request enters write service, start transaction.

T2 UPDATE price SET v=88 WHERE id=1001, commit.

T3 DEL cache:price:1001 executed after transaction commit — if deleted inside transaction, concurrent read during uncommitted state could repopulate stale value, causing new-DB-old-cache after commit.

T4 User read request, GET cache:price:1001 misses.

T5 SELECT price (reads replica, may briefly still see 99).

T6 SET cache with TTL. Note T5 may read replica stale value — this is the residual window delayed double delete addresses; with small replica lag (<10ms) usually acceptable.

Critical ordering rule: delete cache after DB transaction commits . Inside transaction causes two issues: (1) rollback deletes cache unnecessarily (minor), (2) uncommitted read repopulates stale value, commit leaves new-DB-old-cache (major). One line of code placement separates correctness from incident.

2.2 Failure Path: Delete Fails — Three-Layer Safety Net

Layer 1 : Delete call with timeout (e.g., 100ms) + one immediate retry — most network blips recover.

Layer 2 : Still fails → publish to MQ delayed message (e.g., 2s later), consumer idempotently deletes.

Layer 3 : MQ consumer repeatedly fails → dead-letter queue + alert, human intervenes; cache expires naturally via TTL in the meantime.

Core idea: promote "failed delete" from a local exception to a trackable, replayable, bounded task . Incident two's local thread pool was fatal because it gave failed deletes no name — no retry, no record, no endpoint.

2.3 Binlog Fallback Timeline

With Canal added, write path gains an async shadow chain: business commit → MySQL writes binlog → Canal (pseudo-replica) pulls (tens of ms) → parses row change → publishes to MQ → consumer computes cache key from "table+PK" rule and deletes. This chain prioritizes durability over speed : it catches business-code missed deletes, failed delete calls, and ops manual DB updates (direct UPDATE bypassing business layer — binlog chain is the only eye that sees them). Post-failure, cache relies on TTL (typically 1–5 min) to converge.

Binlog chain operational gotchas:

Checkpoint loss : Canal restart with corrupted checkpoint file — either replay history (idempotent delete harmless, but bulk replay spikes DEL) or jump to current (intervening changes may miss delete, TTL converges).

DDL changes : Schema changes break parser; consumer must explicitly ignore DDL events and alert.

Single-table hotspot : Flash-sale stock table thousands of changes/sec; MQ floods with same-table messages. Consumer must aggregate by PK (delete once per PK per time window) or delete volume equals change volume. Each manifests as silent fallback failure — hence mandatory end-to-end latency monitoring (see 4.5).

2.4 Concurrent Read-Write Race: The Tiny-Probability Window

Classic counterexample: read R at T0 cache miss → T1 reads DB gets old 99 → write W commits 88 and deletes cache → T2 read R repopulates SET cache=99. Cache holds stale 99 until TTL expiry. Trigger requires three conditions: read misses before write, read repopulation slower than full write+delete, write delete occurs exactly before repopulation. Probability already low; further requires read-round-trip > write+delete latency. With small replica lag and fast reads, practically never occurs. This derivation — not hand-waving — underpins community consensus: "window acceptable, TTL fallback suffices". To truly close it, delayed double delete (500ms async second delete) adds one async delete per write.

Core Source Code: Five Key Implementations

3.1 Cache-Aside Read Path: Repopulation Must Carry TTL

public PriceDTO getPrice(long id) {
    String key = "price:" + id;
    String v = redis.get(key);
    if (v != null) {
        if ("NULL".equals(v)) // cache-null placeholder
            return null;
        return parse(v);
    }
    // fallback to DB
    PriceDTO dto = priceDao.findById(id);
    if (dto == null) {
        // cache-null placeholder, short TTL
        redis.set(key, "NULL", 120);
        return null;
    }
    // repopulation MUST carry TTL: final safety net for eventual consistency
    redis.set(key, toJson(dto), 300);
    return dto;
}

3.2 Write Path: Post-Commit Delete + Three-Layer Fallback

@Transactional
public void updatePrice(long id, int v) {
    priceDao.update(id, v);
    // CRITICAL: register to run AFTER commit
    TransactionSynchronizationManager
    .registerSynchronization(afterCommit(() -> evict(key(id))));
}

void evict(String key) {
    try {
        redis.del(key);  // with 100ms timeout
    } catch (Exception e) {
        // Layer 1 failure: send to MQ delayed retry
        mq.send("cache-evict-retry", key, 2);
    }
}
// Consumer side idempotent: duplicate DEL harmless
// 16 retries exhausted -> dead-letter + alert

Two details: Spring's TransactionSynchronization.afterCommit guarantees post-commit delete; MQ consumer needs no deduplication — DEL is naturally idempotent, unlike business updates.

3.3 Delayed Double Delete Async Implementation: Delay in Scheduler, Not Business Thread

// WRONG (incident three):
// updateDb(); del(key);
// Thread.sleep(500); del(key);

// CORRECT: async scheduling, business thread returns immediately
public void doubleDelete(String key) {
    redis.del(key);          // immediate delete
    delayScheduler.schedule(
        () -> redis.del(key),
        500, TimeUnit.MILLISECONDS);
}
// Even safer: send MQ delayed message
// delay2s consumer deletes again, auto-retries
// thread pool only as fallback, size by write peak estimate

3.4 Canal Rule-Based Delete: Table-Row to Cache-Key Mapping

// After consuming binlog change message:
// change { table:"price", id:1001 }
Map<String,String> rules = Map.of(
    "price", "price:{id}",
    "sku_stock", "stock:{id}");
void onRowChange(Row r) {
    String tpl = rules.get(r.table);
    if (tpl == null) return; // unrelated table
    String key = tpl.replace("{id}", String.valueOf(r.pk));
    redis.del(key); // idempotent, failure handled by MQ retry
}
// Aggregated cache (detail page multi-table) -> same PK
// change in any source table maps to same cache key

Rule table also makes "which caches driven by which tables" explicit configuration — new table adds one rule line, not another buried delete call in business code. Missed-delete problem eliminated structurally, not by code review.

3.5 Versioning Scheme: Completely Close Repopulation Window

If business has zero tolerance for the tiny-probability stale repopulation window but wants to avoid delayed double delete, versioning is cleaner: DB maintains monotonically increasing version (timestamp or dedicated version column) for critical fields. On successful write, push latest version to a separate version key ( version:price:1001 = 17). Cache value carries its written version. Read path compares before returning — if cached version lags version key, treat as dirty, discard, and re-fetch. Repopulation window sealed. Cost: one extra version check per read (can merge via MGET into single round-trip).

// Read path: cache value carries version, stale -> discard
String v = redis.get(key);
if (v != null) {
    Cached<PriceDTO> c = parse(v);
    long ver = redis.incrBy("ver:" + id, 0);
    if (c.version >= ver) return c.data;
    redis.del(key); // dirty: stale repopulation residue
} // else fallback, repopulate with current ver

// Write path: update DB, then advance version, then delete cache
priceDao.update(id, v);
redis.incr("ver:" + id);  // version first
redis.del(key);             // then delete cache

Order matters: advance version then delete cache . Reverse order allows a read to repopulate with old version yet pass validation. Versioning suits high-consistency, high-read-cost hot keys (e.g., promo pricing); normal keys short TTL cheaper.

Advanced Topics: Seven Accounts to Settle Upfront

4.1 TTL: The Ultimate Fallback — How to Set

All consistency schemes rely on "inconsistency window has an upper bound"; TTL is that bound. Two heuristics:

Business tolerance × 2 as baseline (tolerate 1 min stale → TTL 2 min minimum).

Add jitter to avoid mass simultaneous expiry (base + random).

Too short → frequent fallback (hit rate and cost collapse); too long → fallback ineffective (inconsistency drags). Counter-intuitive: shorter TTL lowers demand on delete reliability — 30s TTL cache fails delete entirely, inconsistency lasts only 30s. Thus "high consistency + high throughput" viable combo is often ultra-short TTL + accept higher fallback rate , not complex double-delete stack. Total cost accounting required.

4.2 Master-Slave Lag: Extra Window in Read-Replica Architecture

In "write-master read-replica", inconsistency window adds replica replication lag: write commit → delete cache done, read fallback hits replica, if replication incomplete, stale value repopulated. Window = replica lag (typically 10–100ms, peaks to seconds). Countermeasures in order:

Force read-master on cache miss (only affects miss fraction, master load increment controllable, simplest).

Delayed double delete (delay > replica lag, async).

Semi-sync replication (replica ack before commit ack, shrinks window at write latency cost).

Most teams stop at #1 — miss fallback is only a few percent of reads, master handles it easily.

4.3 Aggregated Cache Multi-Table Consistency: Who Triggers Invalidation

Product detail cache aggregates price, stock, title across N tables. Core question: "who triggers invalidation?" Two modes:

Multi-source trigger : every source table's binlog maps to same cache key (see 3.4 rule table), any table change deletes entire detail cache. Cost: stock changes 100/sec → detail cache deleted 100/sec → hit rate collapses.

Split cache : separate volatile (stock, price) from stable (title, images) into different keys; page assembles from multiple reads. Volatile keys short TTL or no cache direct read; stable keys long TTL.

Criterion: field change frequency delta . Mixing second-level and day-level changing fields in one cache guarantees poor hit rate. Frequency-layer first, then consistency.

4.4 Delete Storm: Batch Write Impact on Cache

Operator bulk updates 10k products, scheduled task refreshes 100k stock rows — each write triggers DEL, delete QPS spikes to tens of thousands. Redis survives, but next minute all those keys suffer cache miss fallback , DB hit by unplanned fallback storm. Isomorphic to cache avalanche, trigger shifts from "mass expiry" to "mass delete". Countermeasures:

Stagger deletes in batch writes (per-row delay tens of ms, or batch delayed messages).

Rate-limit delete ops in write service (e.g., max 2000 DEL/sec, excess queued).

Pre-warm predictable batch changes — compute new values ahead, after delete pre-warm task immediately repopulates, skipping raw fallback.

4.5 Monitoring & Acceptance: Making Consistency Visible

Consistency bugs are nasty: "no errors, data silently wrong". Three monitoring handles:

Sampling comparison : scheduled job randomly samples N keys, compares cache vs DB (for critical fields like price, stock), reports mismatch count. Only proactive way to detect silent inconsistency.

Delete pipeline metrics : DEL failure count, MQ retry backlog, dead-letter depth — any non-zero triggers alert.

Binlog pipeline latency : Canal-to-consumer end-to-end P99; >5s indicates fallback layer failing.

Acceptance must include chaos test: artificially break delete (firewall block Redis write port 30s), verify MQ retry catches, cache eventually converges, monitoring fires. If any of three checks fails, solution not production-ready.

4.6 Solution Selection Decision Tree

Condense full article into a decision tree, follow sequentially:

Q1: How long can business tolerate stale data? Minutes → Cache-Aside + TTL, stop (80% scenarios end here).

Q2: Seconds? → Cache-Aside + binlog fallback + short TTL; if read-replica add "miss read master".

Q3: Measured stale-repopulation issues? → Async delayed double delete, delay > replica lag.

Q4: Must read always see latest? → Challenge requirement (usually means "1s stale unacceptable"), if truly strong → Write-Through or lock serialization, accept throughput cost.

Each tier down doubles complexity and failure surface — selection discipline: "stop at acceptable tier, don't deploy 100% solution for 1% scenario".

4.7 Stale Data in Production: Five-Step Triage

On "page shows old price" report, follow fixed order, don't jump to blame solution:

Confirm reality — directly GET cache key vs DB, verify true inconsistency vs frontend/CDN cache (many "cache inconsistency" are actually browser/CDN layer, exclude first).

Check key TTL — TTL -1 (immortal key) means repopulation missed TTL or someone SET without expiry; immortal key + delete failure = infinite window.

Inspect delete pipeline — DEL failures, MQ retry backlog, dead-letter contents; this step locates 80% of stale-data incidents.

Audit write paths — does this key's every write point use unified wrapper? Any new write code bypassing delete? Binlog rule table missing this table's mapping?

Replay window — correlate binlog timestamps with cache repopulation timestamps, reconstruct exact sequence, determine root cause (delete failure, stale repopulation, replica lag), then reinforce accordingly.

Learning Path: From Usage to Production Hardening

Week 1 : Get Cache-Aside right — post-commit delete, repopulation with TTL, delete with timeout + one retry. Write unit test for each of the three details.

Weeks 2–3 : Build binlog fallback — Canal + MQ + rule-based delete, validate "missed delete caught" and "delete failure retried" scenarios, wire dead-letter alert.

Weeks 4–6 : Establish monitoring — sampling comparison job, delete pipeline metrics, binlog latency, run one "break write 30s" chaos drill.

Deep dive : Read two implementations — Spring Cache's CacheInterceptor (how annotations weave into read/write paths) and Canal's instance config model (checkpoint and HA management). After understanding framework boundaries, decide if custom consistency component needed.

High-Frequency Interview Questions (with Summary Answers)

20 questions covering all article points, answers lead with conclusion then reasoning, ready for interview phrasing. Key Q&As:

Q1: Why update DB then delete cache? Only survivor among four combos; smallest defect surface, TTL covers residual race.

Q2: Why delete not update? Delete idempotent & order-insensitive; lazy computation avoids wasted work; aggregated caches shift recomputation to read.

Q3: What inconsistency remains in Cache-Aside? Classic race: read miss → read old → write commit+delete → read repopulates old. Three conditions, tiny probability, TTL self-heals. Async delayed double delete closes it.

Q4: Delayed double delete solves what? How to implement safely? Fixes stale repopulation & read-replica stale read. Delay > replica lag + read RTT; second delete async (thread pool or delayed message), never sleep in business thread; does not fix delete failures — those need MQ retry chain.

Q5: Delete fails — what to do? Three layers: timeout+immediate retry → MQ delayed retry (idempotent) → dead-letter+alert, TTL converges meanwhile. Core discipline: failed delete must have a destination, never vanish silently.

Q6: Why delete after transaction commit? Inside transaction: rollback causes unnecessary delete (minor); uncommitted read repopulates stale, commit leaves new-DB-old-cache (major). Spring TransactionSynchronization.afterCommit guarantees order.

Q7: Binlog (Canal) solves what, at what cost? Zero intrusion (new write points auto-covered), catches ops manual DB updates, failure handling via MQ retry/dead-letter. Cost: Canal+MQ components, second-level latency, checkpoint/HA ops. Role: Cache-Aside fallback, not sole mechanism.

Q8: Master-slave read-write split adds what consistency issue? Cache miss fallback hits replica, may read stale during replication lag. Fix gradient: miss read master (most common), delayed double delete, semi-sync replication.

Q9: Write-Through vs Cache-Aside essence? Write-Through: app only touches cache, cache sync-writes DB, all ops serialized through one layer — natural strong consistency but double write latency, custom cache layer needed. Cache-Aside: app manages both, read/write paths separate, high throughput, eventual consistency only. True strong consistency → former; else Cache-Aside sufficient.

Q10: Aggregated cache (multi-table detail) consistency? Frequency-layer first: split volatile (stock, price) from stable (title, images) into separate keys. Same-key multi-source invalidation via binlog rule table; but high-frequency table kills hit rate — layering is prerequisite.

Q11: Batch write (10k price updates) cache impact? Delete storm: 10k DEL spike, then 1min concentrated fallback = unplanned fallback storm (avalanche isomorphic). Fixes: stagger deletes (batch delayed messages/rate-limit), rate-limit DEL in writer, pre-warm predictable batches.

Q12: How to monitor consistency? Three handles: sampling comparison (only proactive silent-inconsistency detector), delete pipeline metrics (non-zero = alert), binlog pipeline P99 latency (>5s = fallback failing). Plus chaos drill: break write 30s, verify retry convergence & alert.

Q13: TTL role in consistency? How to set? TTL = hard upper bound of inconsistency window, final safety net. Repopulation must carry TTL. Baseline: tolerance × 2, add jitter. Paradox: shorter TTL relaxes delete reliability demand; "high consistency + high throughput" often = ultra-short TTL + accept higher fallback rate.

Q14: How to evaluate which tier a scenario needs? Decision tree: minutes → Cache-Aside+TTL (80% stop); seconds → +binlog fallback, read-replica add miss-read-master; measured repopulation issues → async delayed double delete; true strong consistency → challenge requirement, confirm → Write-Through or locks, accept throughput cost. Each tier doubles complexity — stop at acceptable tier.

Q15: Can cache-DB consistency be strong? Conventional means only eventual: two independent stores, no atomic protocol. Strong consistency requires serializing reads/writes (distributed lock or Write-Through single-layer serialization), throughput drops to single-digit QPS. In practice, calibrate requirement — most "strong" means "1s stale unacceptable", satisfied by Cache-Aside+binlog+short TTL at 10x lower cost.

Q16: Local cache (Caffeine) + Redis multi-level consistency? Local cache cross-process unreachable by DEL. Options: ultra-short local TTL (seconds, accept stale window), Redis change broadcasts invalidation (MQ/Redis pub-sub to all instances), or version check (local key carries version, read compares with Redis, stale discards). Local cache only for extremely hot, stale-tolerant data.

Q17: When is operate-cache-before-DB correct? Rare: (1) counter/rate-limiter where cache is primary, DB async backup; (2) read-your-writes — update cache first guarantees writer sees own write, but need invalidation to prevent others reading dirty. General business data: stick to update-DB-then-delete-cache.

Q18: How to guarantee cache delete not lost with MQ? Producer: send on delete failure (or post-commit), send acknowledgment + local message table prevent send loss. Consumer: manual ack, auto-retry (RocketMQ 16x), exhaust → dead-letter+alert. Consumer logic idempotent (DEL naturally idempotent). Pipeline endpoint must have visible dead-letter, no "delete fails then vanishes".

Q19: Versioning vs delayed-double-delete+short-TTL tradeoff? Versioning: DB increments version per write, cache value carries version, read validates version mismatch → discard & re-fetch. Eliminates repopulation window entirely. Cost: extra version read per request (merge via MGET). Suits high-consistency, high-read-cost hot keys; normal keys short TTL cheaper.

Q20: Pre-launch validation scenarios for consistency solution? Four categories: (1) normal concurrent read/write, sampling comparison zero inconsistency; (2) artificial DEL failure (block Redis write port 30s), verify MQ retry convergence, dead-letter alert fires; (3) missed delete (bypass business direct UPDATE DB), verify binlog fallback catches; (4) batch write storm, verify rate-limit & pre-warm effective. Validate defenses when healthy; failure time is acceptance.

One-Page Cheatsheet

Four combos, one survivor : Update DB then delete cache (Cache-Aside) — delete idempotent & race-immune, lazy fallback saves wasted compute, aggregated values delegated to next read.

Three non-negotiable details : Delete after commit (afterCommit), repopulation with TTL (final safety net), failed delete must have destination (timeout+retry → MQ → dead-letter+alert).

Delayed double delete three keys : Delay > replica lag, async execution (no business-thread sleep), only treats stale repopulation not delete failures.

Binlog fallback three unique powers : Zero intrusion prevents missed deletes, catches ops manual DB updates, failure handling via MQ replayable.

Selection decision tree four tiers : Minutes → Cache-Aside+TTL / Seconds → +binlog / Measured repopulation → +async double delete / True strong → Write-Through or locks (challenge requirement first).

Monitoring three handles : Sampling comparison proactive detection, delete pipeline metrics non-zero alert, binlog latency P99. One-sentence closure: Consistency isn't eliminating inconsistency, it's giving the inconsistency window an acceptable ceiling and a visible fallback chain.

Summary & Next Episode Preview

Starting from three real incidents, the article derives a complete cache consistency methodology: Cache-Aside chosen from four combos, three reasons delete beats update, two ordering details (post-commit delete, repopulation with TTL), three-layer delete fallback, Canal binlog rule-based fallback, correct delayed double delete, and tolerance-tiered decision tree. Core mindset in one sentence:

The value of a solution lies not in eradicating inconsistency, but in making the inconsistency window controllable, giving failed deletes a destination, and making silent errors visible to monitoring.

Next episode tackles the classic and most pitfall-prone distributed systems topic — Redis Distributed Locks and the Redlock Controversy : why SET NX EX isn't enough, watchdog renewal implementation, Redlock's five-step flow, and what the famous Martin Kleppmann debate actually argued.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

RedisBinlogTTLCache ConsistencyCanalVersioningCache-AsideDelayed Double Delete
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.