Handling Interface-Level Failures: Degradation, Circuit Breaking, Rate Limiting, and Queuing

In real‑world business operations, interface‑level faults occur when the system remains up but business functionality degrades, manifesting as slow responses, timeouts, or database connection errors. These faults are usually caused by high load, such as database slow queries exhausting resources.

The root causes are divided into internal reasons (bugs, endless loops, memory leaks) and external reasons (attacks, traffic spikes, third‑party service slowdowns).

The core idea for handling such faults is to prioritize core business and the majority of users, using four main techniques: degradation, circuit breaking, rate limiting, and queuing.

1. Degradation

Degradation reduces or disables certain features to keep core functionality alive. It can be implemented via system back‑door URLs or an independent degradation system with permission management.

2. Circuit Breaking

Circuit breaking stops calls to an unhealthy external service, preventing cascading failures. It requires a unified API call layer for monitoring and threshold design (e.g., 30% of requests slower than 1 s within a minute triggers the break).

3. Rate Limiting

Rate limiting controls the amount of traffic the system accepts, either by total request count or by time‑window limits. It can be request‑based (total or per‑minute limits) or resource‑based (limiting connections, file handles, threads, queue size). Common algorithms include fixed window, sliding window, leaky bucket, and token bucket.

3.1 Fixed and Sliding Windows

Fixed windows count requests in discrete intervals, while sliding windows use overlapping intervals to avoid “boundary” spikes.

3.2 Leaky Bucket

Requests enter a bucket (queue) and are processed at a constant rate; excess requests are dropped when the bucket is full.

3.3 Token Bucket

Tokens are added to a bucket at a controlled rate; a request proceeds only if a token is available, allowing limited bursts.

4. Queuing

Queuing is a variant of rate limiting that buffers incoming requests (often using a message queue like Kafka) and processes them when capacity permits, improving user experience compared to outright rejection.

The article concludes by summarizing these four methods and encouraging readers to apply them to improve system resilience.