How Distributed Clusters Achieve Load Balancing: Principles and Practices

Distributed Cluster Load Balancing

1 What is a Distributed Cluster

To understand the concept of a distributed cluster we first clarify the two terms “cluster” and “distributed”. Using a restaurant analogy, a single kitchen with one chef represents a single node. When the business grows, hiring two more chefs creates a cluster that shares tasks such as washing, preparing, and cooking vegetables.

Increasing kitchen productivity can be done in two ways: (1) add more chefs – expanding the cluster, or (2) introduce a production line with finer‑grained division of labor, similar to the diagram below.

The production line embodies the "divide‑and‑conquer" idea: a large task is split into many small tasks, each processed more efficiently. Distributed systems adopt this idea by breaking a big problem into multiple geographically separated nodes.

Key differences:

Distributed systems aim to shorten the execution time of a single task.

Clusters increase the number of tasks completed per unit time.

From a deployment perspective, a distributed system spreads business across locations, while a cluster concentrates several servers to serve the same business.

Each node in a distributed system can itself be a cluster.

A cluster is not necessarily distributed.

In an ideal distributed cluster, each node is itself a clustered group, and those clusters are also distributed.

2 What is Load Balancing

A single server’s capacity is limited by its hardware scalability. When handling massive user requests, a load balancer is introduced to aggregate multiple ordinary servers into a system that can process high concurrency.

Common load‑balancing technologies include DNS, LVS, Nginx, HAProxy, reverse proxy, and commercial solutions like F5. The following diagram classifies these techniques.

Web load balancing typically works in two stages:

DNS‑based (query‑type) routing : The DNS server returns the IP of the nearest service based on the user’s network (e.g., telecom, netcom).

Forwarding (proxy) routing : After DNS directs the request to a nearby IP, a forwarding node (LVS or Nginx) distributes the traffic to multiple real servers (RS).

These techniques are illustrated in the following color‑coded diagram.

Blue indicates DNS‑based user access (GSLB), green shows forwarding‑based load balancing (LVS), and red depicts internal node‑to‑node load balancing within a distributed system.

3 Distributed Cluster Load Balancing

Applying web load‑balancing ideas to distributed nodes is possible, but backend developers often avoid it because different scenarios emphasize different priorities, leading to varied balancing strategies.

Two fundamental goals for web services are simplicity (users access services via easy‑to‑remember domain names) and security (hide internal IPs and network topology). Achieving these goals usually introduces additional nodes (DNS, LVS, reverse proxy), which increase system complexity.

For distributed systems, adding many balancing nodes is undesirable. Fewer nodes mean simpler, more reliable communication. A simple communication pattern—UDP request + UDP acknowledgment—reduces TCP connection overhead.

Key considerations when designing load balancing for distributed systems:

Identify the primary balancing point: request balancing vs. data balancing.

Request balancing aims for each real server to handle a similar number of requests.

Data balancing aims for each real server to store or process a similar amount of data.

Remember: the goal is not to balance for the sake of balancing, but to achieve the desired performance and reliability with the simplest architecture.