Mastering Cache Strategies: From Local Maps to Distributed Grids

Data Classification for Caching

Before adding a cache layer, classify the data to decide which parts benefit most from caching. Two orthogonal dimensions are useful:

Change frequency

Static data – rarely changes (e.g., dictionary tables).

Quasi‑static data – very low change rate (e.g., department hierarchy, national administrative divisions).

Intermediate state data – reusable computed results, local copies of configuration values.

Access frequency

Hot data – accessed very frequently.

Read‑heavy data – read operations far outnumber writes.

Data that is both infrequently modified and frequently read is an ideal candidate for caching because it reduces repeated expensive reads.

Cache Loading Strategies

Full load at application startup – populate the cache with all required entries before the service begins handling requests.

Lazy loading – load entries on demand.

Synchronous load – on a cache miss, read the value from the backing store, put it into the cache, and return it to the caller.

Asynchronous delayed load – return the current cache content (even if empty) and trigger a background thread to fill or refresh the entry.

Strategy 1: If the cache entry is missing, start an async task that loads the data once.

Strategy 2: Run a periodic or condition‑driven async task that keeps the cache up‑to‑date.

Cache Expiration Policies

FIFO or LRU eviction – remove the oldest or least‑recently‑used entries when the cache reaches its size limit.

Fixed‑time expiration – each entry has a TTL (time‑to‑live) after which it becomes stale.

Business‑driven weighted expiration – assign different TTLs based on business importance or update frequency.

Local Cache Implementations

Simple Map cache

public static final Map<String, Object> CACHE = new HashMap<>();
CACHE.put("key", "value");

Guava Cache

Cache<String, String> cache = CacheBuilder.newBuilder()
    .maximumSize(1024)
    .expireAfterWrite(60, TimeUnit.SECONDS)
    .weakValues()
    .build();
cache.put("word", "Hello Guava Cache");
System.out.println(cache.getIfPresent("word"));

Spring Cache (annotation‑driven) Core annotations: @Cacheable – cache the result of a method call. @CachePut – update the cache without affecting method execution. @CacheEvict – remove entries when data changes.

Annotations work only when the call goes through Spring’s AOP proxy; direct calls bypass the cache.

Drawbacks of Local Cache in a Cluster

Memory duplication – each node stores its own copy, increasing overall RAM usage.

Long‑lived objects increase GC pressure in the JVM.

Cache maintenance competes with business threads for CPU and I/O resources.

Remote Cache Solutions

Redis – an open‑source, in‑memory key‑value store written in ANSI C. Supports persistence, replication, clustering and client libraries for many languages.

Memcached – a distributed high‑performance caching system (BSD‑licensed) originally built for LiveJournal.

In‑Memory Data Grids

Hazelcast – provides distributed maps, queues, and compute APIs with automatic data partitioning.

Apache Ignite – offers an in‑memory data fabric with SQL, key‑value, and compute capabilities.

Common Cache Problems and Mitigations

1. Cache Penetration

Massive concurrent requests for keys that do not exist bypass the cache and overload the database.

Cache a placeholder (e.g., null or a sentinel) for missing keys.

Use a Bloom filter or RoaringBitmap to pre‑check key existence before hitting the DB.

Adopt delayed asynchronous loading so that the cache becomes the source of truth.

Bloom Filter Example (Guava)

public static void main(String[] args) {
    BloomFilter<CharSequence> filter = BloomFilter.create(
        Funnels.stringFunnel(Charsets.UTF_8),
        1_000_000, // expected insertions
        0.01);      // false‑positive rate
    System.out.println(filter.mightContain("abcdefg")); // false
    filter.put("abcdefg");
    System.out.println(filter.mightContain("abcdefg")); // true
}

RoaringBitmap Example

public static void test3() {
    Roaring64NavigableMap map = Roaring64NavigableMap.bitmapOf(3, 4, 5, 90);
    boolean contains = map.contains(3);
    long rank = map.rankLong(3);
    System.out.println(rank);
    System.out.println(contains);
}

2. Cache Breakdown (Stampede)

When a hot key expires, many threads may simultaneously miss the cache and flood the database.

Guard the refresh operation with a global mutex (e.g., a distributed lock) so that only one thread repopulates the entry.

Use delayed asynchronous loading to make the cache the authoritative source and avoid synchronous bursts.

3. Cache Avalanche

Simultaneous expiration of a large number of keys can cause a sudden surge of traffic to the backend, potentially crashing it.

Stagger TTLs so that expirations are uniformly distributed over time.

Distribute hot data across multiple cache nodes.

Deploy master‑slave replication or clustering for high availability.

Implement circuit‑breaker and rate‑limiting to protect downstream services.

Use a local fallback cache to absorb short‑term spikes.

Bloom Filter Fundamentals

A Bloom filter allocates an m -bit array initialized to 0. Each element is hashed by k independent hash functions; the resulting positions are set to 1. To query, the same k bits are checked: if any is 0 the element is definitely absent; otherwise it is probably present with a configurable false‑positive rate.

Bitmap Basics

A bitmap uses a single bit to represent the presence of an element, achieving high space efficiency. In Java an int holds 32 bits, so one int can represent 32 distinct values. For a value M, the array index is M / 32 and the bit position is M % 32.

Typical applications include:

Fast sorting of large integer sets by setting bits and then scanning the array in order.

Deduplication of billions of integers using two‑bit states (00 = absent, 01 = once, 11 = multiple).

Rapid membership queries where memory footprint must be minimal.