How to Achieve Ordered Task Execution with a Custom Java Thread Pool

Introduction

In product‑oriented systems that handle intensive task scheduling, a thread pool is often preferred over middleware because it reuses a fixed set of worker threads, improving throughput and response time. However, standard thread pools do not guarantee strict task order, which is essential for certain business scenarios.

Background

The Designer component of the "Instant Consumption Call Center 2.0" abstracts reusable business capabilities into modular components that are orchestrated via a workflow engine. To keep the system lightweight and avoid external dependencies, Designer relies on a thread pool for dense task dispatch.

Problem

Each call generates a unique call‑business ID, and all events related to that call (answer, hold, end, etc.) must be processed sequentially to maintain business consistency and user experience. In a distributed setup, routing ensures the same ID reaches the same instance, but a native Java thread pool processes tasks from different IDs concurrently, causing out‑of‑order execution.

Analysis

Simple solutions such as a single‑thread pool or using CountDownLatch enforce order but sacrifice concurrency and increase coupling. Existing third‑party solutions were either unavailable or introduced a central scheduler that becomes a bottleneck, adds scheduling overhead, and only indirectly guarantees order.

Solution Design

We built a custom thread pool called OrderedTaskExecutor with four key components:

Hash‑based Task Dispatch When a task is submitted, the business ID is hashed to compute an index that determines the target slot (queue). All tasks sharing the same ID are routed to the same slot, providing deterministic routing.

Isolated Task Queues The executor maintains a list of BlockingQueue instances, one per slot (size configurable via slotSize ). Each queue can have a maximum capacity to prevent unbounded growth.

Event‑Driven Worker Scheduling Each slot has an AtomicBoolean run flag. When a task arrives in an idle slot, a worker thread is launched to process that queue. The thread runs until the queue is empty, then clears the flag. New tasks later reactivate the slot.

Support for Asynchronous Tasks Methods such as executeOrderedAsync(bizId, callable) allow submission of Callable tasks, returning a CompletableFuture with the result.

Execution Flow

Workers repeatedly call processTasks(i) for their slot, polling tasks with a timeout. If a task is retrieved, processSingleTask(i, orderedTask) executes the runnable or callable, handling timeouts, statistics, and exceptions. When the queue stays empty, the worker exits and resets the slot flag.

Example

Consider three tasks: executor.execute("1234", taskA), executor.execute("1234", taskB), and executor.execute("5678", taskC). Tasks A and B hash to slot 0 and are processed sequentially by the same worker, while task C hashes to slot 1 and runs in parallel on a separate worker.

Performance

In a stress test on a 12‑core, 24 GB machine with slotSize=180 and a native thread pool configured to (core = 180, max = 180, keep‑alive = 120 s, queue = 10 000), the system handled 12 836 calls per minute, each with eight events, achieving a stable throughput of about 1 700 tasks per second.

Conclusion & Future Work

The custom OrderedTaskExecutor resolves out‑of‑order execution in the call‑center scenario while retaining the concurrency benefits of a thread pool. Future improvements include dynamic resource allocation based on load, comprehensive performance monitoring, and migration to cloud‑native or microservice architectures.