Using Weighted Round Robin to Satisfy Per‑Platform Rate Limits in Java

Scenario

I needed to query a downstream system for a batch of records belonging to different platforms. The downstream service enforces a strict limit of one request per second, but platform A later asked for a slower pace (e.g., one request every 6 seconds).

Initially I solved the problem with a naive Thread.sleep(1000) loop:

// Pseudo‑code: fetch data for a platform and call the downstream API once per second
public void processDataWithDelay() {
    List<DataObject> dataList = database.fetchData("A");
    for (DataObject data : dataList) {
        try {
            Thread.sleep(1000); // 1 s delay
            downstreamService.query(data.getId());
        } catch (Exception e) {
            // handle exception
        }
    }
}

When platform A required a 6‑second interval, I simply replaced the constant with a configurable value, but this made the whole pipeline 5‑6 times slower because the platforms were processed sequentially.

Idea

Instead of blocking the entire pipeline, I interleaved requests from other platforms during the idle time of platform A. By treating the selection of the next platform as a load‑balancing problem, a weighted round‑robin algorithm can allocate more slots to faster platforms while respecting the slower interval of platform A.

The weighted round‑robin approach ensures that the required gap for platform A is maintained, yet the remaining seconds are used to send requests for platforms B, C, etc., achieving better overall throughput without violating the downstream rate limit.

Implementation

Below is a concise Java implementation of a weighted round‑robin scheduler:

public class WeightedRoundRobin {
    private List<Server> servers = new ArrayList<>();
    private int index = -1;
    private int remaining = 0;

    static class Server {
        String name;
        int weight;
        int current;
        Server(String name, int weight) {
            this.name = name;
            this.weight = weight;
            this.current = weight;
        }
    }

    public void addServer(String name, int weight) {
        servers.add(new Server(name, weight));
        remaining += weight;
    }

    public String getNext() {
        if (remaining == 0) resetWeights();
        while (true) {
            index = (index + 1) % servers.size();
            Server s = servers.get(index);
            if (s.current > 0) {
                s.current--;
                remaining--;
                return s.name;
            }
        }
    }

    private void resetWeights() {
        for (Server s : servers) {
            s.current = s.weight;
            remaining += s.weight;
        }
    }
}

And a simple main method to demonstrate the request sequence:

public static void main(String[] args) {
    WeightedRoundRobin wrr = new WeightedRoundRobin();
    wrr.addServer("A平台", 1);
    wrr.addServer("B平台", 2);
    wrr.addServer("C平台", 3);
    for (int i = 0; i < 12; i++) {
        System.out.println("Request " + (i + 1) + " -> " + wrr.getNext());
    }
}

The output matches the expected pattern where two consecutive A‑platform requests are spaced exactly 6 seconds apart, while the intervening slots are filled by B and C requests, fully utilizing the idle time.

This solution demonstrates a pragmatic trade‑off: avoid over‑engineering a perfect load‑balancer, but apply a well‑known algorithm to meet real‑world rate‑limit constraints efficiently.