Designing Elastic Microservice Architecture for Traffic Peaks

The speaker, a senior expert from a leading education technology company and author of the go-zero framework, shares over 20 years of experience in high‑performance computing, backend development, and microservice architecture.

When transitioning to microservices, the key challenges are ensuring high availability under increasing traffic and effective service governance. The talk outlines a five‑part approach: data splitting, cache design, microservice layering, governance capabilities, and an overall overview.

Data Splitting : Clear data boundaries are essential; each service should own its database and communicate via RPC. The presenter discusses practical examples such as separating user, product, order, and logistics data, and mentions using minimal joins to reduce DB load.

Cache Design : To handle high concurrency, the system uses MySQL/Mongo clusters for caching and addresses three critical issues—cache penetration, cache breakdown, and cache avalanche—by implementing short‑lived caches, single‑flight controls, and staggered expiration with a 5% standard deviation.

Additional cache strategies include unique‑index‑based retrieval to ensure a single cache copy per record and automated migration across cache clusters for seamless scaling.

Microservice Layering : Services are exposed via gRPC, with service discovery via etcd and load balancing using a p2c EWMA algorithm. The architecture follows a power‑of‑two‑choices approach for multi‑zone resilience.

Governance Capabilities : The system enforces timeout coordination, controlled retries with capacity budgeting, and adaptive circuit breaking that adjusts based on request success rates rather than fixed thresholds.

Elastic Design & High Availability : Incoming requests undergo concurrency control and rate limiting, followed by adaptive load shedding and auto‑scaling. CPU usage thresholds trigger probabilistic request dropping to protect Kubernetes from overload, ensuring critical services like login remain available during traffic surges.

The overall framework integrates these mechanisms to provide a unified, error‑resistant solution for building highly available, elastic microservices, especially in high‑traffic online education scenarios.