Designing High‑Availability Backend Interfaces

Preface

As a backend developer, creating service interfaces is routine, but ensuring high availability— the ability of a system to handle and mitigate risks— is far from trivial. This article discusses the considerations for building highly available interfaces.

What Is High Availability?

High availability simply means a system’s capability to respond to and avoid risks.

Why Pursue High Availability?

Development errors can cause production incidents.

Hardware components (CPU, memory, disk, network) may fail.

Critical user flows (e.g., registration) can be disrupted if an interface crashes.

Large‑scale events (e.g., Double‑Eleven, 618) can overwhelm order services, harming GMV.

Other unknown factors may arise.

Therefore, we must design for high availability.

Key Points of High Availability

Four factors guide the design: Dependence (few dependencies), Probability (low failure probability), Scope (limited impact), and Time (short impact duration).

Principles for High‑Availability Interface Design

1. Control Dependencies

Minimize dependencies and avoid strong coupling. Use weak dependencies where possible, such as asynchronous processing for coupon issuance during user registration.

2. Avoid Single Points of Failure

Deploy services across multiple data centers.

Retain previous release versions for quick rollback.

Ensure at least two people understand each business service.

Use master‑slave setups for databases and caches.

3. Load Balancing

Distribute traffic across multiple nodes (e.g., Nginx, JSF) to prevent bottlenecks and mitigate hotspot issues in caches like JIMDB.

4. Resource Isolation

Physically separate service deployments and shard data across databases/tables to contain failures.

5. Rate Limiting

Apply flow control to protect both the service itself and its downstream dependencies, using existing JSF rate‑limiting capabilities or custom modules.

6. Service Circuit Breaking

Use circuit breakers (e.g., Hystrix, DUCC) to downgrade strong dependencies to weak ones when downstream services degrade.

7. Asynchronous Processing

Convert synchronous operations to asynchronous ones (e.g., using MQ for coupon distribution during traffic spikes) to reduce load and risk.

8. Degradation Plans

Prepare fallback strategies for critical interfaces, sacrificing non‑core functionality to keep core services running during incidents.

9. Gray Release

Gradually roll out new services to a subset of users, monitor performance and stability, and expand or rollback based on feedback.

10. Chaos Engineering

Proactively inject failures (e.g., via the Tai Shan platform) to discover hidden weaknesses and develop mitigation plans.

Conclusion

The article concludes with an invitation to join an architecture discussion group and lists several promotional links to related projects and resources.