Traffic Governance and High‑Availability Strategies for Microservices

High‑availability systems aim for the "three‑high" goals: high performance, high availability, and easy scalability. Availability is calculated as MTBF / (MTBF + MTTR) * 100%, emphasizing the need to extend MTBF and shorten MTTR.

Traffic governance ensures balanced and efficient data flow, serving three main purposes: optimizing network performance, guaranteeing service quality, and providing fault tolerance and security.

1. Circuit Breaker – Traditional circuit breakers have three states (Closed, Open, Half‑Open) to prevent cascading failures. Google SRE’s adaptive throttling circuit breaker uses client‑side request and acceptance counters to compute a rejection probability p = max(0, (requests - K*accepts) / (requests + 1)) , where K adjusts aggressiveness.

2. Isolation – Various isolation strategies (dynamic/static, read/write, core, hotspot, user, process, thread, cluster, and data‑center isolation) partition resources or traffic to limit the impact of a single service failure.

3. Retry – Retry logic includes synchronous and asynchronous modes, with back‑off strategies (linear, jittered, exponential, exponential‑jitter). Proper retry limits, windows, and error‑type filtering avoid retry storms.

4. Degradation – When overload persists, services can downgrade non‑critical functionality automatically or manually, balancing user experience against system load.

5. Timeout – Timeout policies (fixed or EMA‑based dynamic timeout) prevent long‑running requests from exhausting resources. Timeout propagation across RPC calls ensures downstream calls respect remaining time budgets.

6. Rate Limiting – Client‑side and server‑side rate limiting (token bucket, leaky bucket, sliding window) protect services from traffic spikes and control user behavior.

The article also provides a pseudo‑code example of exponential back‑off with jitter used by gRPC:

/* pseudo code */
ConnectWithBackoff()
  current_backoff = INITIAL_BACKOFF
  current_deadline = now() + INITIAL_BACKOFF
  while (TryConnect(Max(current_deadline, now() + MIN_CONNECT_TIMEOUT)) != SUCCESS) {
    SleepUntil(current_deadline)
    current_backoff = Min(current_backoff * MULTIPLIER, MAX_BACKOFF)
    current_deadline = now() + current_backoff + UniformRandom(-JITTER * current_backoff, JITTER * current_backoff)
  }

By combining these mechanisms, a microservice architecture can remain robust under varying network conditions and load, achieving sustained high performance, availability, and scalability.