Mastering Load Balancing: From Round Robin to Consistent Hashing in Java

Preface

Load balancing appears in virtually every high‑availability stack, such as micro‑services, sharding, middleware (MQ, Redis, MyCat, Nginx, ES), cloud computing, cloud scheduling, and big data.

Load‑balancing strategies are mainly divided into static and dynamic categories:

Static scheduling algorithms: dispatch requests based solely on pre‑configured policies.

Dynamic scheduling algorithms: adjust dispatch based on real‑time conditions (network, CPU load, disk I/O, etc.).

This article focuses on several commonly used and efficient load‑balancing algorithms.

Basic Load‑Balancing Algorithms

Fundamental algorithms such as round‑robin, random, and weighted are simple to implement.

Round‑Robin Algorithm

The round‑robin algorithm distributes requests sequentially across the server list, looping back to the first server after reaching the last one.

Java implementation:

public class Servers {
    public static List<String> SERVERS = Arrays.asList(
        "44.120.110.001:8080",
        "44.120.110.002:8081",
        "44.120.110.003:8082",
        "44.120.110.004:8083",
        "44.120.110.005:8084"
    );
}

public class RoundRobin {
    private static AtomicInteger requestIndex = new AtomicInteger(0);
    public static String getServer() {
        int index = requestIndex.get() % Servers.SERVERS.size();
        String server = Servers.SERVERS.get(index);
        requestIndex.incrementAndGet();
        return server;
    }
}

public class Test {
    public static void main(String[] args) {
        for (int i = 1; i <= 10; i++) {
            System.out.println("Request " + i + ": " + RoundRobin.getServer());
        }
    }
}

Advantages: simple implementation, high dispatch efficiency, evenly spreads load, and easily scales horizontally.

Disadvantages: cannot consider server hardware differences, and cannot guarantee session affinity because requests are not bound to the same server.

Applicable scenarios: clusters with identical hardware and stateless read‑only workloads.

Random Algorithm

The random algorithm selects a server at random for each request.

Java implementation:

public class Random {
    static java.util.Random random = new java.util.Random();
    public static String getServer() {
        return Servers.SERVERS.get(random.nextInt(Servers.SERVERS.size()));
    }
}

Advantages: often combined with weighted strategies; disadvantages: cannot evenly distribute load and does not support session affinity.

Weighted Algorithm

Weighted algorithms assign a weight to each server; higher weight means more requests are directed to that server. They are usually combined with a base algorithm (e.g., round‑robin or random).

Random weighted implementation example:

public class Servers {
    public static Map<String, Integer> WEIGHT_SERVERS = new LinkedHashMap<>();
    static {
        WEIGHT_SERVERS.put("44.120.110.001:8080", 17);
        WEIGHT_SERVERS.put("44.120.110.002:8081", 11);
        WEIGHT_SERVERS.put("44.120.110.003:8082", 30);
    }
}

public class RandomWeight {
    static java.util.Random random = new java.util.Random();
    public static String getServer() {
        return Servers.WEIGHT_SERVERS.get(random.nextInt(Servers.WEIGHT_SERVERS.size()));
    }
}

Round‑robin weighted implementation example:

public class RoundRobinWeight {
    private static AtomicInteger requestCount = new AtomicInteger(0);
    public static String getServer() {
        int weightTotal = 0;
        for (Integer w : Servers.WEIGHT_SERVERS.values()) weightTotal += w;
        int index = requestCount.get() % weightTotal;
        requestCount.incrementAndGet();
        for (Map.Entry<String, Integer> e : Servers.WEIGHT_SERVERS.entrySet()) {
            if (e.getValue() > index) return e.getKey();
            index -= e.getValue();
        }
        return null;
    }
}

Both examples suffer from uneven distribution when the request count is small.

Smooth Weighted Round‑Robin Algorithm

This algorithm solves the uneven distribution problem of the simple weighted round‑robin by using a dynamic weight that accumulates each round.

Java implementation:

public class Servers {
    public static Map<String, Integer> WEIGHT_SERVERS = new LinkedHashMap<>();
    static {
        WEIGHT_SERVERS.put("44.120.110.001:8080", 3);
        WEIGHT_SERVERS.put("44.120.110.002:8081", 2);
        WEIGHT_SERVERS.put("44.120.110.003:8082", 1);
    }
}

public class Weight {
    private String server;
    private Integer weight;
    private Integer currentWeight;
    public Weight(String server, Integer weight, Integer currentWeight) {
        this.server = server; this.weight = weight; this.currentWeight = currentWeight;
    }
    // getters and setters omitted for brevity
}

public class RoundRobinWeight {
    private static Map<String, Weight> weightMap = new HashMap<>();
    private static int weightTotal = 0;
    static { sumWeightTotal(); }
    public static void sumWeightTotal() {
        for (Integer w : Servers.WEIGHT_SERVERS.values()) weightTotal += w;
    }
    public static String getServer() {
        if (weightMap.isEmpty()) {
            Servers.WEIGHT_SERVERS.forEach((s, w) -> weightMap.put(s, new Weight(s, w, 0)));
        }
        for (Weight w : weightMap.values()) w.setCurrentWeight(w.getCurrentWeight() + w.getWeight());
        Weight max = null;
        for (Weight w : weightMap.values()) {
            if (max == null || w.getCurrentWeight() > max.getCurrentWeight()) max = w;
        }
        max.setCurrentWeight(max.getCurrentWeight() - weightTotal);
        return max.getServer();
    }
}

The algorithm distributes requests proportionally to the configured weights and is widely used in Dubbo, Ribbon, Nginx, and Zookeeper.

Consistent Hashing Algorithm

Consistent hashing ensures that the same client (or key) is always mapped to the same server, which is essential for stateful services such as sessions or distributed caches.

Core concept: a hash ring of 2³² points; each server is placed on the ring based on the hash of its IP. A request’s hash (e.g., client IP) walks clockwise to the first server.

To avoid uneven load caused by server‑IP clustering, virtual nodes are introduced: each real server maps multiple virtual nodes onto the ring, smoothing the distribution.

Java implementation using a TreeMap as the ring:

public class Servers {
    public static List<String> SERVERS = Arrays.asList(
        "44.120.110.001:8080",
        "44.120.110.002:8081",
        "44.120.110.003:8082",
        "44.120.110.004:8083",
        "44.120.110.005:8084"
    );
}

public class ConsistentHash {
    private static final int VIRTUAL_NODES = 160;
    private static TreeMap<Integer, String> virtualNodes = new TreeMap<>();
    static {
        for (String ip : Servers.SERVERS) {
            virtualNodes.put(getHashCode(ip), ip);
            for (int i = 0; i < VIRTUAL_NODES; i++) {
                int hash = getHashCode(ip + i);
                virtualNodes.put(hash, ip);
            }
        }
    }
    public static String getServer(String clientIP) {
        int hash = getHashCode(clientIP);
        SortedMap<Integer, String> tail = virtualNodes.tailMap(hash);
        int key = tail.isEmpty() ? virtualNodes.firstKey() : tail.firstKey();
        return virtualNodes.get(key);
    }
    private static int getHashCode(String key) {
        final int p = 1904390101;
        int hash = (int)1901102097L;
        for (int i = 0; i < key.length(); i++) {
            hash = (hash ^ key.charAt(i)) * p;
        }
        hash += hash << 13;
        hash ^= hash >> 7;
        hash += hash << 3;
        hash ^= hash >> 17;
        hash += hash << 5;
        return Math.abs(hash);
    }
}

Consistent hashing is commonly used for distributed caching (e.g., Memcached, Redis Cluster) and can also be applied to request routing when session affinity is required.

Least‑Active Algorithm

This dynamic algorithm selects the server with the smallest number of active requests, automatically balancing load based on real‑time activity.

Java sketch (simplified):

public class LeastActive {
    // Assume Servers.attenuator() updates each server's active count.
    public static String getServer() {
        // Find servers with minimal active count, then apply weight/random selection.
        return selectedServerIP;
    }
}

In production, frameworks like Dubbo and Ribbon provide full implementations that also consider server weights.

Fastest‑Response (Best‑Available) Algorithm

This algorithm pings all servers concurrently and chooses the one that responds first, ensuring the request is handled by the most responsive node.

Java implementation using CompletableFuture:

public class ResponseTime {
    static ExecutorService pool = Executors.newFixedThreadPool(Servers.SERVERS.size());
    public static String getServer() throws InterruptedException {
        List<CompletableFuture<String>> futures = new ArrayList<>();
        for (Server s : Servers.SERVERS) {
            futures.add(CompletableFuture.supplyAsync(s::ping, pool));
        }
        CompletableFuture<Object> any = CompletableFuture.anyOf(futures.toArray(new CompletableFuture[0]));
        any.thenAccept(ip -> System.out.println("Fastest node: " + ip));
        Thread.sleep(3000); // wait for async tasks
        return (String) any.get();
    }
}

The method selects the server with the lowest latency, which is useful for latency‑sensitive services.

Conclusion

The article covered static algorithms (round‑robin, random, weighted, consistent hashing) and dynamic algorithms (least‑active, fastest‑response). While sophisticated algorithms can improve load distribution and session affinity, they also add computational overhead. In ultra‑high‑traffic scenarios with homogeneous hardware, the simplest round‑robin often yields the best performance.