Mastering Service Degradation: Keep Your System Available Under Heavy Load

What Is Service Degradation?

Service degradation means temporarily disabling non‑essential features when a system is under extreme load, so that core functions remain available. The idea is similar to limiting visitor numbers in a crowded tourist site and cutting less important activities to focus resources on critical services.

Service Level Definition (SLA)

SLA (Service Level Agreement) is a key metric for judging whether a performance test is abnormal. It measures the guaranteed uptime of a service. Cloud providers often target six‑nine availability (99.9999%).

Six‑nine translates to about 31 seconds of downtime per year, which is considered extremely reliable.

Degradation Handling Techniques

Fallback Data

Default values : Provide safe defaults that do not cause data errors, e.g., inventory set to 0.

Static values : Return static content or a friendly error page when a page or API fails.

Cache : Use the last cached response if the fresh data cannot be retrieved.

Rate‑Limiting Degradation

Set a maximum QPS threshold for each request type. When traffic exceeds the threshold, reject excess requests with a friendly message such as “system busy, please try later.” This protects core services during traffic spikes.

Timeout Degradation

Define a timeout for remote calls. If a call exceeds the timeout and the feature is not core, degrade it by hiding optional content (e.g., product recommendations) while keeping the main shopping flow functional.

Failure Degradation

If a downstream service fails (network, DNS, HTTP error), return default or cached data, or serve a static fallback page.

Retry and Automatic Handling

Client‑side high availability: configure multiple service endpoints.

Microservice retry: use frameworks like Dubbo’s retry mechanism.

API retry: after exceeding retry attempts, mark the service as degraded and perform asynchronous health checks.

Web‑side retry button or automatic retry for better user experience.

Automatic retries must be limited and idempotent.

Degradation Switch

Operators can manually toggle a switch to disable problematic services. The switch value can be stored locally or in external configuration stores such as databases, Redis, or Zookeeper, allowing rapid activation of degradation policies.

Read Degradation

When a read path fails (distributed cache, RPC, DB), the system can fall back to lower‑level caches or static data. Common strategies include:

Temporary read switch : downgrade to cache or static content.

Temporary read block : block the read entry or service entirely.

The typical read flow is: access‑layer cache → application‑layer local cache → distributed cache → RPC/DB. By placing switches at the access or application layer, the system can automatically bypass unavailable components, which is suitable for scenarios with relaxed consistency requirements.

Write Degradation

For high‑write workloads, the system can temporarily write to an in‑memory store (e.g., Redis) and later synchronize to the database, sacrificing immediate strong consistency for availability. This approach is common in flash‑sale or order‑creation scenarios, where writes are queued asynchronously.

Similarly, user reviews can be downgraded to asynchronous processing or limited exposure when the volume is too high.

Front‑End Degradation

When backend services are unavailable, front‑end code can use local cache or fallback data to keep the UI functional. In low‑consistency scenarios (e.g., flash sales), dummy data may be displayed.

JavaScript Degradation

Embed a degradation flag in JavaScript to stop sending requests once a threshold is reached.

Access‑Layer Degradation

Use Nginx + Lua or HAProxy + Lua to filter invalid requests before they reach services, providing an early degradation point.

Application‑Layer Degradation

Configure feature toggles in the application. For example, Spring Cloud’s Hystrix can automatically degrade a service based on timeout settings, providing circuit‑breaker and fallback mechanisms.

Segment Degradation

If certain page fragments fail to load (e.g., product listings), the system can skip those fragments and replace them with alternative content, ensuring the rest of the page remains usable.

Pre‑Embedding (Pre‑warming)

Static data can be pre‑loaded onto client devices before a major event (e.g., a shopping festival) to reduce network load during the peak.