How Eureka Handles Millions of Requests: In-Memory Registry & Multi-Level Caching Explained

Problem Origin

Eureka is a critical component in Spring Cloud architecture, acting as the service registration center for all microservice discovery and registration. Beginners often wonder how many instances of Eureka Server are needed, how much load it can bear, and whether it can sustain large‑scale traffic.

Core Questions

How does Eureka store the addresses and ports sent by services during registration?

What is the frequency at which services pull the registry from Eureka Server?

How do services retrieve the registry?

What load does a system with hundreds of services and thousands of machines place on Eureka Server?

How does Eureka Server handle tens of millions of daily requests?

Client Pull and Heartbeat Mechanism

Each Eureka Client sends a request to the Eureka Server every 30 seconds to fetch the latest registry changes and also sends a heartbeat every 30 seconds to indicate that the instance is alive.

Example: an inventory service expands from 1 to 3 instances, all registering with Eureka. The order service’s client then pulls the registry every 30 seconds to discover the new instances.

Request Volume Estimation

Assume a large system with 100 services, each deployed on 20 machines (2,000 instances). Each instance makes 2 registry‑pull requests and 2 heartbeat requests per minute, totaling 4 requests per minute. That yields 8,000 requests per minute, about 133 requests per second, roughly 11.5 million requests per day.

Eureka Server Registry Storage Structure

The registry is kept in a pure in‑memory ConcurrentHashMap . The map’s key is the service name (e.g., "inventory-service"), and the value is another map that holds multiple instances of that service.

Structure: Map<String, Lease<InstanceInfo>> Key: instance ID

Value: a Lease object that contains an InstanceInfo (IP, hostname, port) and the last heartbeat timestamp.

Multi‑Level Cache Mechanism

Eureka Server uses a two‑level cache to reduce contention and improve read speed.

ReadOnlyCacheMap: first lookup for registry data.

If missing, ReadWriteCacheMap is consulted.

If still missing, the data is fetched directly from the in‑memory map.

When the registry changes, the in‑memory map is updated and the ReadWriteCacheMap is cleared, while the ReadOnlyCacheMap continues serving existing queries. After 30 seconds, a background thread clears the ReadOnlyCacheMap, forcing the next read to refresh from the updated data.

Summary

By setting a 30‑second pull and heartbeat interval, a large system generates only a few hundred requests per second to Eureka Server.

The pure in‑memory registry ensures these requests are processed with minimal latency.

The multi‑level cache prevents frequent read‑write conflicts, further boosting performance.

Consequently, Eureka can comfortably support a system with thousands of instances and handle daily traffic in the tens of millions without degradation.