How EVCache Uses Cloud‑Native Architecture for Scalable Distributed Caching

Large Internet Applications and Caching

Cache is one of the two pillars of internet technology, and distributed caching systems are the key weapon for large‑scale applications. As data volume and traffic patterns grow, caching resolves performance bottlenecks and enables rapid responses.

Advantages of Cloud Services

Cloud platforms provide several commercial and technical benefits:

Zero upfront infrastructure cost : No need to invest in data centers, racks, power, cooling, or extensive staffing.

Instant elasticity : Resources can be provisioned on demand, reducing risk and operational expenses.

Automation and scripting : Infrastructure can be defined via APIs, enabling repeatable builds and deployments.

Auto‑scaling and proactive scaling : Systems can automatically expand or shrink based on load.

Improved observability : Automated testing and monitoring are integrated throughout the development lifecycle.

Disaster recovery and business continuity : Low‑cost replication across regions ensures rapid failover.

Among cloud providers, AWS is highlighted for its low management overhead, high reliability, and seamless scalability.

EVCache: Cloud‑Native Distributed Cache

EVCache is an open‑source, high‑performance distributed cache built on Memcached and the Spymemcached client, optimized for Amazon EC2. It provides:

Distributed key‑value storage that spans multiple instances.

Cross‑AZ data replication within AWS regions.

Automatic registration and discovery of nodes via Netflix’s naming service.

Keys as non‑empty strings; values may be byte arrays, primitives, or serialized objects (max 1 MB).

Cache‑name namespaces to avoid key collisions.

Typical cache hit rates above 99 %.

Integration with Netflix’s data frameworks (e.g., Cassandra, SimpleDB, S3).

Cross‑Region Replication Mechanism

EVCache replicates data across AWS regions using a Kafka‑based message queue. The replication flow for a SET operation is:

EVCache client sends SET to a local‑region server.

The client also writes metadata (key only) to a Kafka queue.

A regional relay service reads the message.

The relay fetches the corresponding value from the local cache.

The relay forwards a SET request to the relay service in the target region.

The target‑region relay writes the value to its local cache, completing replication.

Subsequent GET requests in the target region read the updated value.

This mechanism applies only to SET operations; DELETE and TOUCH are not replicated.

High‑Availability Design

AWS regions consist of multiple Availability Zones (AZs), each with independent power and networking. EVCache deploys EC2 instances across several AZs, ensuring that a failure in one zone does not affect the others. Consistent hashing spreads data across shards, minimizing impact when individual nodes fail.

Because cache misses that fall back to backend services (e.g., Cassandra, S3) are expensive, EVCache’s high hit rate and low‑cost replication keep overall operational costs low while maintaining linear scalability.

Typical Application Scenario: Netflix Recommendation Service

Netflix relies on EVCache for low‑latency access to user‑specific data such as viewing history, rankings, and personalized recommendations. A typical workflow for fetching similar‑content recommendations is:

A client requests a page that needs a list of similar movies or shows.

The web application queries EVCache; cache hit rates exceed 99.9 %.

If the cache misses, the recommendation service computes similarity.

If the computed data is not in SimpleDB, the service reads from SimpleDB or recomputes it.

The newly computed data is written back to EVCache.

The service returns the response to the client.

EVCache can scale linearly; capacity monitoring allows expansion within a minute and rebalancing within a few minutes.