How Adaptive Load Balancing Can Tame Double‑11 Traffic Peaks

During the Double‑11 shopping festival, request rates can exceed server capacities, causing task queues and service degradation. Adaptive load balancing automatically evaluates service capacity and distributes traffic to keep response times short and overload probability low.

Background of Adaptive Load Balancing

The two main goals are to maintain short response times with low blocking probability, and to keep the algorithm overhead within acceptable CPU and network limits.

Adaptive load balancing continuously assesses system load—whether idle, stable, or busy—and reallocates traffic so the system remains performant without starvation or overload.

Challenge Overview

The 5th Middleware Performance Challenge simulates a consumer that receives requests and forwards them to one of three providers (small, medium, large) with differing CPU, memory, and concurrency limits. The consumer must choose a provider using a load‑balancing algorithm.

Providers are accessed directly via configured URLs; no service registry is used.

Algorithm Design Considerations

Capacity assessment: measure response time and maximum concurrency for each provider, detect bottlenecks, and estimate available capacity.

State maintenance: keep per‑provider capacity metrics and use them to make selection decisions.

Auxiliary interfaces: optional features such as bidirectional communication or provider‑side rate limiting can be leveraged if needed.

The default Random algorithm lacks global state and cannot guarantee optimal distribution in heterogeneous or multi‑dispatcher environments, leading to potential overload.

Evaluation Environment

1 pressure machine (4 CPU / 8 GB), 1 gateway machine (4 CPU / 8 GB), and 3 provider machines (each 4 CPU / 8 GB).

Steps: lock workspace, pull code from Git, build a fat JAR, start providers and consumer, warm up for 30 seconds, run the official test for 1 minute, record successful requests and peak TPS, then stop services, clean Docker images, collect logs, and unlock the workspace.

Conclusion

Adaptive load balancing is essential for handling traffic surges like Double‑11, and designing a robust algorithm involves accurate capacity estimation, efficient state management, and careful evaluation. Participants can apply these principles to achieve high performance in the challenge.