How Java’s DelayQueue Handles Million‑QPS Loads: Performance Test Insights

Background

Earlier performance tests used java.util.concurrent.DelayQueue for a 10‑second order‑cancellation scenario. It was omitted from a broader Java‑and‑Go queue comparison because the queue was perceived to rely on sorting and an array‑based implementation, which was thought to be slow.

After reviewing the goreplay project, which replays traffic based on timestamps, the author decided to reuse the timestamp‑driven approach and implement a message queue using DelayQueue.

Implementation

To store items in a DelayQueue, each element must implement java.util.concurrent.Delayed. The following class provides a minimal implementation used in the benchmark.

static class FunTesterDelay implements Delayed {
    long timestamp;
    String str;
    FunTesterDelay() {
        this.timestamp = 5;
        this.str = DEFAULT_STRING;
    }
    @Override
    public long getDelay(TimeUnit unit) {
        // Return remaining delay in the requested unit.
        return timestamp - Time.getTimeStamp();
    }
    @Override
    public int compareTo(Delayed o) {
        // Returning 0 disables ordering; the queue behaves like a simple FIFO.
        return 0;
    }
}

The benchmark creates a fixed thread pool (default 20 threads) and a DelayQueue<FunTesterDelay>. Each worker repeatedly adds a new FunTesterDelay instance to the queue.

static int THREAD_COUNT = 20;
static int OPERATIONS = 100_000;
static DelayQueue<FunTesterDelay> delays = new DelayQueue<>();

public static void main(String[] args) {
    RUNUP_TIME = 0;
    new Concurrent(new FunTester(), THREAD_COUNT,
        "DelayQueue performance test").start();
}

private static class FunTester extends FixedThread {
    FunTester() { super(null, OPERATIONS, true); }

    @Override
    protected void doing() throws Exception {
        delays.add(new FunTesterDelay());
    }

    @Override
    FunTester clone() { return new FunTester(); }
}

Test Model

The test reuses the static load model employed in previous Java & Go queue benchmarks: a fixed number of worker threads each performing a fixed number of add (or add‑then‑remove) operations. No external I/O is involved, so the measured metric is pure queue throughput.

Results – Add Only

Throughput is reported as QPS (operations per second) in ten‑thousands. The values show how total QPS scales with thread count while per‑thread QPS declines.

1 thread – 540 k QPS (single‑thread 540 k)

5 threads – 1.94 M QPS (single‑thread 380 k)

10 threads – 3.25 M QPS (single‑thread 320 k)

20 threads – 4.5 M QPS (single‑thread 220 k)

50 threads – 3.15 M QPS (single‑thread 60 k)

100 threads (10 × 10⁴ ops) – 3.33 M QPS (single‑thread 30 k)

100 threads (20 × 10⁴ ops) – 2.58 M QPS (single‑thread 25 k)

200 threads – 2.83 M QPS (single‑thread 14 k)

Results – Add & Remove

When each added element is immediately removed, the throughput is comparable, indicating that internal ordering does not dominate the cost.

1 thread – 480 k QPS

5 threads – 1.23 M QPS (single‑thread 240 k)

10 threads – 1.97 M QPS (single‑thread 190 k)

20 threads – 2.46 M QPS (single‑thread 120 k)

50 threads – 2.84 M QPS (single‑thread 50 k)

100 threads (10 × 10⁴ ops) – 2.55 M QPS (single‑thread 25 k)

100 threads (20 × 10⁴ ops) – 2.83 M QPS (single‑thread 28 k)

200 threads – 2.83 M QPS (single‑thread 14 k)

Conclusion

The java.util.concurrent.Delayed implementation can sustain multi‑million QPS, easily exceeding the target of 500 k QPS per process for the FunTester workload. As the number of threads increases, total QPS plateaus around 2.5 – 3 M while per‑thread efficiency drops, likely due to CPU and memory‑bandwidth limits. Keeping the queue short and avoiding excessive thread counts yields the best balance between latency and throughput.