How Eureka Server Handles Millions of Requests with In-Memory Registry & Multi-Level Cache

1. Problem Origin

In Spring Cloud architecture, Eureka is a crucial microservice registry. Beginners often ask how many machines to deploy, the access pressure, and whether Eureka can handle large‑scale traffic.

2. Eureka Server’s Registry Storage Structure

Assuming a system with 100 services, each deployed on 20 machines (2000 instances), each instance sends a request to Eureka every 30 seconds to pull the registry and a heartbeat every 30 seconds. This results in about 150 requests per second and roughly 11.5 million requests per day.

The registry is implemented as a

ConcurrentHashMap<String, Map<String, Lease<InstanceInfo>>>

stored entirely in memory. The key is the service name, the value is a map of instance IDs to Lease<InstanceInfo>, which holds instance details and the last heartbeat time.

3. Multi‑Level Cache Mechanism

Eureka uses a read‑only cache (ReadOnlyCacheMap) and a read‑write cache (ReadWriteCacheMap). When a client pulls the registry, it first checks the read‑only cache, then the read‑write cache, and finally the in‑memory data. Updates invalidate the read‑write cache without affecting the read‑only cache, which is refreshed every 30 seconds.

4. Summary

Appropriate pull and heartbeat intervals keep per‑second requests at a few hundred even for large systems.

Storing the registry entirely in memory ensures high‑performance reads and writes.

The multi‑level cache prevents frequent read‑write conflicts, further boosting performance.

Thus, Eureka can comfortably support a large Spring Cloud deployment with daily traffic in the ten‑million range.