How Different Load‑Balancing Strategies Impact Reliability in Distributed Systems

In distributed systems, load balancing is a crucial component that distributes requests to one or more nodes for processing. Load balancing can be hardware‑based (e.g., dedicated appliances like F5) or software‑based (using load‑balancing software or built‑in modules).

Generally, the following common load‑balancing strategies exist:

1. Round‑robin – a classic strategy where each request is assigned a sequential number and dispatched to servers in order. Suitable when all nodes have equal capacity and are stateless. Its drawback is treating all nodes as identical, which may not reflect real‑world conditions. Weighted round‑robin adds a weight per node, but static weights are hard to adjust dynamically.

2. Random – similar to round‑robin but selects a node randomly without numbering. Also assumes equal nodes; weighted random variants exist.

3. Minimum response time – records the time each request takes, computes average response time, and selects the node with the smallest average. It reflects server state better, but the averaging introduces latency, making it unsuitable for rapid response requirements. Variants may use recent samples only.

4. Minimum concurrency – because request processing times vary, simple round‑robin or random can cause uneven connection counts. Minimum concurrency tracks the number of ongoing transactions on each candidate node at the current moment and chooses the node with the fewest. This quickly reflects current load and distributes work more evenly, useful for load‑sensitive scenarios.

5. Hash – used when backend nodes maintain state; requires hashing to keep session affinity. The article does not discuss this further.

Other strategies exist but are not listed here.

Distributed systems face far more complex environments than single‑machine systems—varying networks, platforms, hardware, etc. Errors are inevitable, so fault tolerance and minimizing error cost are essential. The choice of load‑balancing strategy significantly affects reliability.

Consider the following scenario. Completing a request requires invoking four clusters A, B, C, D. Suppose the call to cluster B must be made three times, and cluster B has five servers.

If one server in cluster B fails and other fault‑tolerance mechanisms have not yet taken effect, ideally 4/5 of the requests remain unaffected.

With round‑robin or random, the probability that a single request is sent to a healthy node is 4/5, so the success probability of the whole request is (4/5)³ ≈ 64/125 ≈ 0.512, lower than the ideal 4/5.

Thus, using only these strategies spreads the impact of failures, which is undesirable.

If the minimum concurrency strategy is used, assuming a normal request takes 10 ms and the timeout is 1 s, a healthy node can handle about 100 requests in the timeout window while a failed node can handle only 1. The probability of dispatching to the failed node becomes 1/(100·4+1)=1/401, giving a success probability of (400/401)³ ≈ 99.25 %, higher than 4/5.

More generally, let p be the proportion of failed servers in a cluster. The expected success probability for k sequential calls under round‑robin/random is shown below:

When k=3, the relationship between success rate f(p) and p is illustrated:

The graph shows that as p increases, f(p) drops noticeably, so simple strategies are unsuitable for high‑reliability distributed systems.

If the minimum concurrency strategy is used, with total servers m and degraded service capacity 1/q for a failed node, the total service capacity per unit time is:

Thus the probability of sending a request to a healthy node is:

The overall success rate is this probability raised to the k‑th power:

When q=10 and k=3, the success‑rate curve f(p) is:

The graph indicates that for small p (e.g., 0–0.4), the success rate does not decline sharply, and the system can handle multiple node failures as long as each node can sustain the load.

When p is constant, i.e., a certain fraction of machines are already faulty, the analysis still applies.

When p=0.1 and k=3, the success‑rate curve f(q) is:

The graph suggests that setting the timeout to about ten times the normal service time is sufficient.

If failures are detected quickly by the client (e.g., client notices a misconfigured backend node, q<1), assuming q=0.1 and k=3, the relationship becomes:

In this case, failure probability rises dramatically even with a small faulty proportion, requiring additional detection mechanisms.

To mitigate this, a protection mechanism removes a backend node after it fails consecutively beyond a threshold. Assuming a 1 % chance of spurious failure, a removal threshold of 9, q=0.1, and 5 available nodes, the probability of repeatedly selecting the same faulty node is (2/7)⁹ ≈ 0.001 %, which can be ignored.

In practice, client‑side concurrency may stay low, which does not reflect server‑side load and can cause imbalance when client concurrency is small.

Therefore, the minimum concurrency strategy is unsuitable for client‑side load balancing with low client concurrency; random selection is often used instead.

In real distributed systems, a single node failure can increase pressure on other nodes, potentially degrading their performance, and the interactions are difficult to capture with simple equations.