Traffic Governance and High Availability in Backend Systems: Circuit Breakers, Isolation, Retries, Timeouts, and Rate Limiting

High availability (HA) in computer systems is measured by Availability = MTBF / (MTBF + MTTR) * 100%.

Traffic governance ensures balanced and efficient data flow, acting like a nutritionist for system health.

Purpose : improve network performance, guarantee service quality, provide fault tolerance, enhance security, and reduce costs.

Techniques :

Circuit breakers (traditional three‑state and Google SRE adaptive throttling).

Isolation (static/dynamic, read/write, user, tenant, process, thread, cluster, data‑center).

Retry strategies (synchronous, asynchronous, back‑off algorithms, retry windows, hedging).

Degradation (automatic and manual fallback).

Timeout control (fixed, EMA‑dynamic, propagation across services).

Rate limiting (client‑side and server‑side, token bucket, leaky bucket, sliding window).

Implementations often use gRPC exponential back‑off, Sentinel, or custom sliding‑window logic.

ConnectWithBackoff()
  current_backoff = INITIAL_BACKOFF
  current_deadline = now() + INITIAL_BACKOFF
  while (TryConnect(Max(current_deadline, now() + MIN_CONNECT_TIMEOUT)) != SUCCESS) {
    SleepUntil(current_deadline)
    current_backoff = Min(current_backoff * MULTIPLIER, MAX_BACKOFF)
    current_deadline = now() + current_backoff + UniformRandom(-JITTER*current_backoff, JITTER*current_backoff)
  }

Combining these mechanisms helps build systems that are performant, highly available, and easy to scale.