Prevent Service Avalanches: Configuring Circuit Breaker & Connection Limits in Envoy Gateway

In microservice architectures, service availability and stability are critical; uncontrolled downstream latency or failures can cause a "cascade" or avalanche effect that brings down the entire system.

Core Concepts

The two key fault‑tolerance mechanisms are Circuit Breaker and Connection Limit . A circuit breaker temporarily stops sending requests to a downstream service when its failure rate exceeds a threshold, allowing the service time to recover. Connection limits control the maximum number of concurrent connections between a client and a backend service, preventing any single client from monopolising resources.

Circuit Breaker Configuration Example

Before applying the configuration, the hey tool is used to generate 100 concurrent requests with a 5‑second artificial delay, showing normal response times.

$ hey -c 100 -n 100 --host www.simple.com http://172.139.20.19:30874/ping?delay=5s
Summary:
  Total:        5.0312 secs
  Slowest:      5.0244 secs
  Fastest:      5.0053 secs
  Average:      5.0129 secs
  Requests/sec: 19.8759
[200] 100 responses

Tip: The default circuit‑breaker threshold (1024) is not reached, so requests do not overflow; all requests are proxied upstream and both Envoy and the client wait 5 seconds.

Apply the circuit‑breaker policy with the following YAML:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: circuitbreaker-for-route
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: simple
  circuitBreaker:
    maxPendingRequests: 5
    maxParallelRequests: 10

After applying, verification with hey shows that most requests are rejected (503) because the circuit breaker limits the load:

$ hey -c 100 -n 100 --host "www.simple.com" http://172.139.20.19:30874/ping?delay=5s
Summary:
  Total:        5.0218 secs
  Slowest:      5.0216 secs
  Fastest:      0.0010 secs
  Average:      0.5096 secs
  Requests/sec: 19.9130
[200] 10 responses
[503] 90 responses

Tip: Setting maxPendingRequests to 0 triggers an overflow circuit‑breaker for all requests.

Connection Limit Configuration Example

Before configuring, hey generates 10 concurrent requests at 1 QPS for 10 seconds (total 100 requests) to establish a baseline.

$ hey -c 10 -q 1 -z 10s --host www.simple.com http://172.139.20.19:30874/ping
Summary:
  Total:        10.0131 secs
  Slowest:      0.0117 secs
  Fastest:      0.0021 secs
  Average:      0.0040 secs
  Requests/sec: 9.9869
[200] 100 responses

Apply a connection‑limit policy via a ClientTrafficPolicy:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
  name: connection-limit-ctp
  namespace: default
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: simple-gw
  connection:
    connectionLimit:
      value: 5

Verification after applying the limit shows many requests failing with connection‑reset errors, confirming that the limit is enforced.

$ hey -c 10 -q 1 -z 10s --host "www.simple.com" http://172.139.20.19:30874/ping
Summary:
  Total:        10.0120 secs
  Slowest:      0.0123 secs
  Fastest:      0.0023 secs
  Average:      0.0042 secs
  Requests/sec: 9.9880
[200] 57 responses
Error distribution:
  [23] Get "http://172.139.20.19:30874/ping": EOF
  ... (multiple connection reset errors) ...

Conclusion

In modern cloud‑native architectures, circuit breakers and connection limits are essential building blocks for system stability. Using Envoy Gateway, these capabilities can be declared declaratively without modifying application code, providing a straightforward way to enhance resilience.