Handling Interface-Level Failures: Degradation, Circuit Breaking, Rate Limiting, and Queuing

In real‑world business operations, interface‑level failures often do not cause a full system crash or network outage, yet they degrade user experience with slow responses, timeouts, or error messages such as “cannot connect to database.”

The root causes fall into two categories: internal reasons (bugs, infinite loops, slow database queries, memory exhaustion) and external reasons (hacker attacks, traffic spikes from promotions, heavy third‑party requests).

Core Principle

The key idea, similar to multi‑active disaster recovery, is to prioritize core business functionality and the majority of users.

1. Degradation

Degradation reduces or disables certain features while keeping the core service alive. Examples include limiting a forum to read‑only mode or temporarily stopping log‑upload endpoints. Two common implementations are a simple back‑door URL that triggers degradation and a dedicated degradation system that provides fine‑grained control and batch operations.

2. Circuit Breaking

Circuit breaking stops calls to an unhealthy external service to prevent cascading failures. It requires a unified API call layer for monitoring and a threshold design (e.g., if more than 30% of requests exceed 1 s within a minute, trigger the circuit). When the circuit is open, the service returns an error immediately.

3. Rate Limiting

Rate limiting controls the amount of traffic the system can accept. It can be request‑based (total count or per‑time‑window limits) or resource‑based (limiting connections, threads, queue length, CPU usage). Common algorithms include fixed and sliding windows, token bucket, and leaky bucket, each with distinct advantages and drawbacks.

4. Queuing

Queuing is a variant of rate limiting that buffers excess requests instead of rejecting them, allowing users to wait. Implementations typically rely on an external message queue (e.g., Kafka) and involve a queuing module, a scheduler that pulls tasks when capacity is available, and a service module that processes the tasks.

By applying these four strategies—degradation, circuit breaking, rate limiting, and queuing—systems can maintain core functionality and provide a better user experience even under high load or partial failures.