Process Cache vs Cache Service: Choosing Redis and Ensuring Data Consistency

We distinguish two types of cache: in‑process (internal) cache and external cache services like Redis or Memcached. As applications evolved from monolithic to multi‑instance and micro‑service architectures, external cache services have become increasingly popular.

In‑process cache stores data inside the web or service process. Historically popular (e.g., .NET's System.Web.Caching), it can be implemented with a locked hashtable or a list, allowing storage of data, HTML fragments, files, or any objects. It reduces database hits by first checking the cache and only querying the database when the data is missing, thus improving efficiency for relatively static, large‑volume data such as system dictionaries or configuration.

However, in high‑traffic, multi‑instance environments, in‑process caching suffers from consistency problems because each instance maintains its own copy of the data. When many nodes modify data independently, keeping the caches synchronized becomes difficult.

To achieve consistency, three common approaches are presented:

1. A single service node notifies other nodes after updating the database and its own cache, prompting them to refresh their caches.

2. Use a message queue (MQ) to broadcast change events; all nodes subscribe to the MQ and update their local caches upon receiving a message.

3. Deploy an independent background process that periodically pulls the latest data from the backend and refreshes the in‑memory cache, sacrificing real‑time consistency for simplicity.

These methods add synchronization overhead and may conflict with the core caching principle of separating hot and cold data; frequent updates diminish cache value.

When to abandon in‑process cache in favor of a cache service:

• The system runs a web cluster with multiple instances where data inconsistency is unacceptable.

• The volume of cached data exceeds the memory capacity of a single process.

• Evaluation of value size, memory, peak QPS, expiration time, hit rate, read/write strategy, key distribution, expiration policy, and consistency requirements indicates a cache service is more appropriate.

Cache Service (Redis) is the most common key‑value cache in modern architectures. Its notable features include:

Support for complex data structures (strings, hashes, lists, sets, sorted sets) enabling use cases such as order lists, user messages, and comment threads.

Persistence options: periodic asynchronous snapshots (semi‑persistent) and Append‑Only File (AOF) for full persistence, though Redis is not intended to replace a primary database.

High availability through native clustering, replication, read/write splitting, and Sentinel for monitoring and automatic failover.

Large storage capacity: strings up to 512 MB; lists, sets, and hashes can hold up to 2³²‑1 elements.

Transactional support with atomic operations, ensuring either all changes succeed or none do, thus preventing data inconsistency.

Practical usage notes highlight common pitfalls such as missing cache entries, cache penetration, and cache avalanche, which the author mentions will be covered in a separate analysis.

Disclaimer: The material is collected from the internet, the copyright belongs to the original authors, and the content reflects personal opinions only. It is provided for learning and discussion purposes.