Service Reliability Essentials: Rate Limiting, Circuit Breaking & Degradation

Service Problems

System operation inevitably encounters issues. Service problems can be viewed from two angles: infrastructure failures (machine crashes, network or data‑center faults) and service‑program issues (slow responses, timeouts, overload, or failures of individual microservices).

Rate Limiting

Rate limiting controls the amount of traffic a service can handle within a given time window, preventing overload and timeouts. When the request volume exceeds the configured limit, excess requests are rejected or delayed.

NGINX implementation

Server‑side token‑bucket or leaky‑bucket algorithms

Circuit Breaking

Circuit breaking protects services by stopping calls to an unhealthy downstream service, similar to a fuse in an electrical circuit. When error rates exceed a threshold, the circuit opens and subsequent calls return immediately (often with a fallback). After a cooldown period, the circuit attempts a half‑open state to test recovery before fully closing.

Open: normal operation, monitoring success/failure rates.

Half‑open: limited trial requests to check recovery.

Closed: service operates normally.

Key components include Hystrix (deprecated), hystrix‑go, resilience4j, and Sentinel.

Degradation (Fallback)

When a service is unavailable, degradation provides graceful fallback responses instead of raw errors. Strategies include timeout fallback, failure‑count fallback, fault fallback (default values, cached data, static pages), and rate‑limit fallback (queue pages, out‑of‑stock messages, error pages).

Timeout degradation: configure timeout and retry mechanisms.

Failure‑count degradation: trigger after a threshold of failed calls.

Fault degradation: use default or cached data when the remote service is down.

Rate‑limit degradation: apply when traffic exceeds limits, directing users to queues or informative pages.

Adaptive Circuit Breaker Algorithm

The adaptive algorithm uses request count, successful count, and a sensitivity factor K to compute a probability p. If p > 0, the circuit opens; otherwise it stays closed. The algorithm dynamically adjusts to traffic patterns, providing self‑healing behavior.

Implementation in Go‑Zero

Go‑zero defines a Breaker interface for circuit breaking. The core implementation resides in GoogleBreaker , which evaluates request statistics and decides whether to allow or reject calls. The token‑bucket example shows how to create a bucket, fill tokens, and check allowance with Allow . The sliding‑window example demonstrates how to track timestamps, purge expired entries, and enforce request limits.

Rate‑Limiting Algorithms

Common metrics include QPS, concurrent connections, response time, HPS, and TPS. Limiting can be applied at the UI (CAPTCHA, IP blacklist), ingress (NGINX limit modules), or service layer (token‑bucket, sliding‑window).

NGINX rate limiting : rate=2r/s with burst control; limit_conn perip and per‑server directives.

Token bucket : tokens generated at a fixed rate; requests consume tokens.

Sliding window : divides a time window into sub‑intervals, counting requests per sub‑interval to smooth spikes.

Degradation Types in Architecture

Degradation can be automatic (timeout, failure‑count, fault, rate‑limit) or manual (UI disabling, delayed processing via message queues, read/write restriction, cache fallback, exception throwing, or tiered resource reduction).