Which Load‑Balancing Strategy Guarantees the Highest Reliability?

In distributed systems, load balancing is crucial for distributing requests to one or more nodes. It can be implemented via hardware load balancers (e.g., F5) or software load balancers installed on servers.

1. Round‑Robin assigns a sequential number to each request and distributes them evenly across nodes, assuming equal capacity and statelessness. Its weighted variant adds per‑node weights but still struggles with dynamic conditions.

2. Random selects a node at random for each request, also treating all nodes as equal; a weighted random version exists but is not detailed here.

3. Minimum Response Time records the average response time of each node and routes requests to the node with the smallest average. Because it relies on averaged data, it may lag behind rapid changes.

4. Minimum Concurrency tracks the current number of active transactions on each node and selects the node with the fewest concurrent requests, offering fast reaction to load spikes and better fault tolerance.

5. Hash is used when backend nodes maintain state; the article does not explore this further.

When a node in a cluster fails, the ideal impact is limited to the proportion of requests handled by that node. Using round‑robin or random, the probability of a request reaching a healthy node is 4/5, leading to a success probability of (4/5)^3 ≈ 0.512, which is lower than the ideal 0.8.

With the minimum concurrency strategy, assuming a normal request takes 10 ms and a timeout of 1 s, the faulty node’s capacity is 1 while each healthy node’s capacity is 100. The probability of routing to the faulty node becomes 1/(4·100+1)=1/401, giving a success probability of (400/401)^3 ≈ 99.25%.

Generalizing, if the fraction of faulty nodes is p, the expected success probability under round‑robin/random is (1‑p)^k, where k is the number of required calls. Under minimum concurrency, the success probability is ((m‑1/q)/(m‑1/q+ p·(1/q)))^k, with m the total nodes and q the degradation factor.

Graphs illustrate how the success rate f(p) declines as p increases for both strategies. When p is small (e.g., ≤0.4), the minimum concurrency strategy maintains high reliability, while round‑robin/random degrades noticeably.

Additional considerations include removing consistently failing nodes after a threshold of failures; for example, with a failure‑removal threshold of 9 and a 1% random failure rate, the chance of mistakenly removing a healthy node is negligible.

However, the minimum concurrency strategy is unsuitable for client‑side load balancing when client concurrency is low, as it may not reflect server‑side load accurately; in such cases, random selection is often used.