How to Load Test Multi‑Row Single Updates with Thread‑Safe Queues in Java

Problem Statement

In load‑testing scenarios where each database row may be updated only once, the test must generate a unique request parameter for every operation. When the data set contains millions of rows, guaranteeing that each concurrent thread receives a distinct value that matches an existing row becomes a scalability challenge.

Solution Overview

Two thread‑safe designs are proposed:

Global queue: Before the test starts, read all required parameters into a single LinkedBlockingQueue. Each worker thread obtains a value by calling take() (or poll() to avoid blocking) and uses it for the request.

Per‑thread queue: Create an independent queue for each thread. Because a thread accesses only its own queue, the queue does not need to be thread‑safe, simplifying the implementation at the cost of additional memory.

Pseudocode

Queue<String> q = Data.init(); // preload all parameters
while (testRunning) {
    String param = q.take(); // or q.poll() to return null when empty
    doRequest(param);
}

The take() method blocks until an element is available, while poll() returns null immediately if the queue is empty, allowing the test to decide how to handle the shortage.

Java Implementation (Global Queue)

public class LoadTestHelper {
    // Shared queue populated once at start‑up
    public static LinkedBlockingQueue<String> msgs = loadQueue();

    private static LinkedBlockingQueue<String> loadQueue() {
        String basePath = new File("").getAbsolutePath();
        List<String> lines = WriteRead.readTxtFileByLine(basePath + "/queue");
        LinkedBlockingQueue<String> q = new LinkedBlockingQueue<>();
        q.addAll(lines);
        logger.info("Queue reloaded with {} entries", lines.size());
        return q;
    }

    public int deleteQ() throws InterruptedException {
        // Re‑load if the queue has been exhausted
        if (msgs.isEmpty()) {
            logger.info("Queue empty, reloading");
            msgs = loadQueue();
        }
        // Example of reading auxiliary configuration
        String basePath = new File("").getAbsolutePath();
        List<String> cfg = WriteRead.readTxtFileByLine(basePath + "/dubbo");
        int threadCount = SourceCode.changeStringToInt(cfg.get(1));
        int param = SourceCode.changeStringToInt(cfg.get(0));

        new Concurrent(new ThreadBase(param) {
            @Override protected void before() {}

            @Override protected void doing() throws Exception {
                // Retrieve a parameter with a short timeout to avoid indefinite blocking
                String msg = msgs.poll(100, TimeUnit.MILLISECONDS);
                logger.info("msg:{}", msg);
                DeleteQueueRequest req = new DeleteQueueRequest();
                req.setMsg(msg);
                req.setTaskTypeEnum(TaskTypeEnum.PUBLISH_PROMU);
                CommonResponse<String> resp = commonDelayQueueService.deleteQueue(req);
                logger.info("deleteQueue response {}", JsonUtil.obj2Json(resp));
            }

            @Override protected void after() {}
        }, threadCount).start();
        return 0;
    }
}

This implementation demonstrates the typical workflow:

Read a large list of unique identifiers from a file (e.g., /queue) and store them in a LinkedBlockingQueue.

Each test thread calls poll(100, TimeUnit.MILLISECONDS) to obtain a value; the timeout prevents a thread from hanging when the queue is temporarily empty.

If the queue becomes empty, the helper reloads it automatically, ensuring the test can continue without manual intervention.

The retrieved value is used to build a request object ( DeleteQueueRequest) and sent via the service client ( commonDelayQueueService).

Because LinkedBlockingQueue is internally thread‑safe, multiple threads can safely call poll or take concurrently without additional synchronization. Pre‑loading millions of entries eliminates the risk of contention or blocking under typical load‑test throughput.

Key Considerations

Memory usage: Loading millions of strings requires sufficient heap space; consider using String interning or compressing the source file if memory becomes a bottleneck.

Queue refill strategy: For extremely long tests, a background producer thread can continuously append new identifiers to the queue, keeping its size within a configurable window.

Blocking vs non‑blocking: Choose take() when the test must wait for a value, or poll() with a timeout when the test should proceed even if no new data is available.