How to Build a Universal Static Data Cache for Microservices with Redis

What is static data

Static data refers to data that changes infrequently, such as vehicle model libraries, basic user information, and basic vehicle information. These datasets are updated rarely, e.g., monthly for model libraries or upon user registration/modification.

Why cache

In user‑or vehicle‑IoT scenarios, model, user, and vehicle data are queried frequently. Storing them in relational databases yields low I/O efficiency under high concurrency. An in‑memory KV cache (e.g., Redis) provides O(1) reads and greatly improves throughput.

Other techniques like read‑write splitting or sharding improve both read and write performance but are less effective when the sole goal is to boost read throughput with limited resources.

General caching mechanism

The proposed mechanism consists of six core components:

Business service : exposes CRUD interfaces for business data (e.g., vehicle service).

Relational database : persists data (SQL Server, MySQL, Oracle, …).

Persistent queue : decouples services and guarantees delivery (RabbitMQ, RocketMQ, Kafka, …).

Cache processor : consumes queue messages and writes them to the cache.

Data‑consistency checker : periodically compares cache and DB, updating stale cache entries.

Cache database (Redis) : a durable in‑memory store, the industry standard for this scenario.

Two external actors are also defined:

Data producer – the source of static data (frontend app, web module, etc.).

Data consumer – services that need the static data (e.g., alarm system).

Why a business service?

Microservice architectures require a single entry point for data operations. A dedicated service avoids duplicated code, simplifies performance tuning, and reduces inconsistency across multiple programs.

Why not use in‑process cache?

In‑process caches are limited by the host’s memory and can cause cache‑snowball effects when many instances restart simultaneously, overwhelming the relational database.

Why Redis?

Redis solves the above problems: it can be deployed independently, clustered for horizontal scaling, persists data across restarts, offers excellent read/write latency, and supports rich data structures.

Why a queue?

The queue decouples the business service from cache updates, enabling asynchronous processing and easier scaling. Persistent queues also protect against data loss caused by network glitches or crashes.

Why persistence for the queue?

Persisting queue messages ensures no data is lost between the service and the cache processor. Confirmation mechanisms are required, though they may introduce duplicate processing and reduce throughput.

Why a data‑consistency checker?

If a service crashes before writing to the cache, or if Redis fails over, the checker detects stale entries and refreshes them, providing a safety net for extreme failure scenarios.

Why not rely solely on cache expiration?

Expiration‑based caching can cause cache‑penetration or snowball effects, especially when TTL is short. Dynamic TTL or pre‑warming strategies add complexity and may still violate the high‑accuracy, low‑latency requirements of static data.

Summary

The mechanism wraps data operations in a business service, publishes changes to a persistent queue, updates Redis via a cache processor, and runs a periodic consistency checker. This design offers high‑concurrency, near‑real‑time reads, decoupled writes, and resilience against data loss, while acknowledging added operational complexity.