Design and Implementation of Proximity Routing and Graceful Startup/Shutdown in Spring Cloud for Multi‑Region Deployments

International site backend services have continuously expanded, supporting dual‑cloud and multi‑region deployments. This growth brings service‑governance challenges such as same‑city multi‑datacenter routing, multi‑region disaster recovery, and graceful service up/down.

This article describes iQIYI’s design, development, and practice for solving the above problems using Spring Cloud.

Proximity routing is needed because, in multi‑datacenter scenarios, a large number of cross‑datacenter or cross‑region network calls significantly increase latency and degrade user experience. Conventional microservice routing strategies do not support multi‑level zone/region decisions, making custom client‑side load balancing essential.

An example deployment includes three providers: one in the same zone (dc1), one in the same region but different zone (dc2), and one in a different region (dc3). Problems arise when the consumer receives the full provider list and performs round‑robin requests, causing unnecessary cross‑zone traffic and lacking proper failover when a provider becomes unavailable.

The required intelligent routing capabilities are:

Consumers must be able to obtain instance lists from all DCs (the registry must support multi‑DC).

Under normal conditions, traffic should stay within the same DC (zone).

If the same‑DC provider is unavailable, traffic should degrade to a provider in the same region; if that is also unavailable, it should fall back to a provider in a different region.

To clarify terminology, the article adopts the AWS model: an Availability Zone (AZ) guarantees <1 ms latency within a single or multiple co‑located data centers; a Region guarantees 2‑5 ms latency across AZs; inter‑Region latency typically ranges from 20‑100 ms.

Existing Spring Cloud capabilities are reviewed:

Netflix Ribbon offers a ZoneAvoidanceRule that can select services based on zone, but it cannot handle region‑level decisions.

Spring Cloud LoadBalancer (newer versions) only provides simple round‑robin, random, and weight‑based strategies, lacking advanced proximity routing.

Therefore, a custom extension is designed with the following steps:

When a provider registers, it calls a service to obtain its own zone and region and includes this metadata in the registration.

When a consumer starts, it also obtains its zone and region.

The consumer first filters instances in the same zone.

If the healthy ratio in the same zone exceeds 50 %, a load‑balancing strategy selects one instance.

If the healthy ratio is below 50 %, the consumer degrades to the same region and repeats the selection logic.

If still insufficient, the consumer falls back to all instances and applies the load‑balancing strategy.

After this routing‑strategy transformation, the client‑side load balancer supports proximity routing and basic disaster‑fallback capabilities.

A case study on the international site shows that after deploying the custom spring-cloud-iqiyi extension, traffic follows channel 1 (all DCs healthy), channel 2 (dc1 removed), and channel 3 (dc1 and dc2 removed), matching the expected behavior.

The article also addresses graceful startup and shutdown issues. Cold‑start instances without pre‑warm can cause timeouts, and killed instances may still receive traffic, leading to 5xx errors. To solve this, the startup process delays service registration until the ApplicationReadyEvent and performs custom pre‑warm logic (e.g., local cache, long‑connection, connection‑pool warm‑up) before registration.

Graceful shutdown is achieved by deregistering the service in the ContextClosedEvent, then waiting a configurable period (default Ribbon 30 s) for consumer server‑list updates before the JVM exits.

Spring Boot 2.3+ provides built‑in graceful shutdown configuration, for example:

server:
  shutdown: graceful   ## enable graceful shutdown
spring:
  lifecycle:
    timeout-per-shutdown-phase: 5s    ## graceful shutdown wait time, default 30s

Overall, the custom extensions greatly improve backend service stability and disaster‑recovery capability. Future work includes open‑sourcing the extensions and adding features such as tag routing and gray deployments.