Common Misunderstandings in Thread Pool Configuration and How to Avoid Them

Introduction

Creating and destroying threads is a heavyweight operation for the operating system, so thread pooling is widely used in many languages. In Java, the thread pool (implemented by Doug Lea) is essential, but misunderstandings about its implementation can lead to incorrect configuration.

Thread‑Pool Execution Logic

When a task is submitted, the pool first checks whether the number of running threads is below corePoolSize ; if not, a new thread is created.

If the core size is reached, the task is placed into the BlockingQueue .

If the queue is full, the pool attempts to increase the number of threads up to maximumPoolSize .

If the maximum size is already reached, the task is rejected according to the rejection policy.

Core Pool

The set of threads whose count is less than or equal to coreSize is called the core pool. Core threads are long‑living and are usually not destroyed, so most submitted tasks should be executed by them.

Misunderstanding About Thread Creation Timing

The Javadoc states: “If fewer than corePoolSize threads are running, try to start a new thread with the given command as its first task.” The word “running” is ambiguous; many interpret it as “already scheduled by the OS”. In fact, a thread is considered running as soon as it is created, even before it gets a CPU slice.

Consequently, if the QPS is very low, the pool may never reach the configured core size, leading to a much smaller actual core pool than expected.

Creation Process

In practice, each submitted task creates a new thread until the core size is reached. The relevant source code is:

public void execute(Runnable command) {
...
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) { // workerCountOf() gets current thread count
if (addWorker(command, true))
return;
c = ctl.get();
}
...
}

BlockingQueue

The BlockingQueue is the second key component of a thread pool. It acts as the buffer in the producer‑consumer model and can absorb traffic spikes, but it is also a frequent source of configuration errors.

Running Model

The pool does not schedule tasks to idle threads; instead, tasks (except the first task of a newly created thread) are always placed into the queue and later consumed by any worker thread chosen by the OS.

Producer Code

public void execute(Runnable command) {
...
if (isRunning(c) && workQueue.offer(command)) { // isRunning() checks pool state
int recheck = ctl.get();
if (!isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
...
}

Consumer Code

private Runnable getTask() {
for (;;) {
...
Runnable r = timed ?
workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
workQueue.take();
if (r != null)
return r;
...
}
}

Buffering Role

If a large number of tasks are submitted simultaneously, they all wait in the queue. When the queue size is smaller than the burst of tasks, excess requests are rejected. In a real‑world service, a small queue (e.g., size 50) can cause rejections even when coreSize is large, as demonstrated by a test with QPS 2000 and 20 ms response time.

Calculating Concurrency

Concurrency (the peak number of simultaneous tasks) is not directly equal to QPS. It must be estimated by multiplying QPS with the average response time, then adding a safety margin. For example, QPS * average response time gives a baseline, and the BlockingQueue size should be set larger than this baseline when possible.

Runtime Considerations

Garbage Collection

GC pauses stop the JVM, but the OS continues to accept I/O, causing request back‑log. The number of requests that accumulate during a GC pause can be approximated by QPS * GC time , so this should be accounted for when sizing the pool.

Business Peaks

When traffic is bursty or driven by scheduled jobs, average QPS is not useful. In such cases, a larger BlockingQueue or a higher maximumPoolSize with sufficient redundancy helps avoid rejections.

Conclusion

Understanding the source code of ThreadPoolExecutor is essential; relying only on secondary articles can leave hidden pitfalls. A solid grasp of the underlying principles enables flexible and reliable thread‑pool configuration for complex backend services.