Thread Pool Misuse in Production: Key Lessons from a Real Incident

Introduction

Thread pool interview questions often cover core parameters ( corePoolSize, maximumPoolSize, workQueue), sizing strategies, and the recommendation to use ThreadPoolExecutor constructors instead of Executors factory methods. However, knowing these "standard answers" does not guarantee correct production usage. This article reconstructs a real production incident to illustrate the gap.

Incident Reconstruction

A developer wrote code that fetched a list of items from a database, then for each item performed a model conversion and an RPC call to get price information. Because the RPC service lacked a batch interface, the code used CompletableFuture.supplyAsync with a custom thread pool ( corePoolSize=10, maximumPoolSize=20) to parallelize the RPC calls. The code worked initially but started timing out after traffic increased.

List<StationData> items = getFromDb();
List<CompletableFuture<AppMapStationData>> completableFutures = items.stream().map(item -> CompletableFuture.supplyAsync(() -> {
    AppMapStationData data = mapper.copy(item);
    // RPC call for price info
    data.setPriceInfo(priceApi.getPriceInfoById(item.getId()))
    return data;
}, apiExecutor)).collect(Collectors.toList());

result = completableFutures.stream().map(e -> {
    return e.get(); // No timeout specified
}).filter(Objects::nonNull).collect(Collectors.toList());

The root cause: Future.get() was called without a timeout. When the RPC response slowed down, the main thread blocked indefinitely on the slowest task. As the DB result set grew, the maximum task duration increased, causing interface timeouts.

Two fixes were required:

Paginate the DB query to limit the number of concurrent tasks.

Always specify a timeout on get().

Timeout Calculation Nuance

Future.get(long timeout, TimeUnit unit)

starts timing when get is called, not when the task was submitted. Example: two tasks A (3s) and B (4s) submitted to a 2-thread pool; calling get(2, SECONDS) sequentially causes A to timeout but B succeeds because B has already run for 2s by the time its get is invoked.

Thread Pool Misuse Examples

(1) Single Thread Pool for All Business Types

A multi-tenant system submitted all tenants' tasks to one shared pool. One tenant's burst caused pool exhaustion, slowing all tenants. Worse, a parent task submitting a child task to the same pool can deadlock: parent tasks occupy all threads while waiting for child tasks that cannot start because no threads are free.

Solution: Use separate named thread pools per business domain. Guava's ThreadFactoryBuilder helps set meaningful prefixes for easier jstack analysis.

ThreadFactory threadFactory = new ThreadFactoryBuilder()
    .setNameFormat(threadNamePrefix + "-%d")
    .setDaemon(true).build();
ExecutorService threadPool = new ThreadPoolExecutor(
    corePoolSize, maximumPoolSize, keepAliveTime,
    TimeUnit.MINUTES, workQueue, threadFactory);

(2) @Async Without Custom Thread Pool

Spring's @Async defaults to SimpleAsyncTaskExecutor, which creates a new thread per invocation (no pooling). Under load this causes thread explosion. The implementation in AsyncExecutionInterceptor wraps the method call in a Callable and submits to an executor; if none is provided, it throws an exception unless a default is set. Best practice: implement AsyncConfigurer, override getAsyncExecutor to return a bounded pool, and reference it via @Async("apiExecutor").

(3) ThreadLocal Leakage in Pooled Threads

Thread pools reuse threads, so ThreadLocal values persist across tasks. Common cleanup pattern uses finally block to call remove(), but Thread.stop() (deprecated but still possible) skips finally. Recommended: use TransmittableThreadLocal (from Alibaba's transmittable-thread-local library) which propagates and cleans context automatically.

Key Thread Pool Points

(1) Why Queue Capacity Cannot Be Dynamically Adjusted

Spring's ThreadPoolTaskExecutor exposes setters for corePoolSize and maximumPoolSize that delegate to the underlying ThreadPoolExecutor, but setQueueCapacity only assigns a field used during initial queue creation ( LinkedBlockingQueue or SynchronousQueue). The queue's capacity field is final, so it cannot be changed after construction. Workaround: implement a custom queue like Meituan's ResizableCapacityLinkedBlockingQueue that removes the final modifier on capacity.

public void setMaxPoolSize(int maxPoolSize) {
    synchronized(this.poolSizeMonitor) {
        if (this.threadPoolExecutor != null) {
            this.threadPoolExecutor.setMaximumPoolSize(maxPoolSize);
        }
        this.maxPoolSize = maxPoolSize;
    }
}

public void setQueueCapacity(int queueCapacity) {
    this.queueCapacity = queueCapacity; // only used in createQueue()
}

protected BlockingQueue<Runnable> createQueue(int queueCapacity) {
    return (queueCapacity > 0 ? new LinkedBlockingQueue(queueCapacity) : new SynchronousQueue());
}

(2) Core Thread Count Formulas

Common rules of thumb:

CPU-intensive: N + 1 (N = CPU cores). Extra thread covers occasional page faults or pauses.

I/O-intensive: 2N. Threads spend most time waiting for I/O, so more threads keep CPU busy.

More scientific formula: Optimal threads = N * (1 + WT/ST) , where WT = wait time (total runtime - compute time), ST = compute time. Higher WT/ST ratio → more threads. VisualVM can measure WT/ST. For CPU-bound tasks WT/ST ≈ 0 → N threads. For I/O-bound WT/ST ≈ 1 → 2N threads.

Linux CPU core count commands:

# Total cores = physical CPUs * cores per CPU
# Physical CPU count
cat /proc/cpuinfo | grep "physical id" | sort | uniq | wc -l
# Cores per physical CPU
cat /proc/cpuinfo | grep "cpu cores" | uniq

(3) Why ParallelStream Is Discouraged

A code review found parallelStream().forEach adding to a non-thread-safe ArrayList, causing missing results. While the bug is the unsafe list, developers often reach for parallelStream assuming it's faster without considering thread safety. Additionally, parallelStream uses the common ForkJoinPool, which shares threads across all parallel streams in the JVM and suffers from the same ThreadLocal leakage issues. For large streams, the common pool can become a bottleneck.

Summary

This article uses a production incident to highlight critical thread pool practices: always bound Future.get() with timeouts, paginate large datasets, isolate thread pools by business domain, configure custom executors for @Async, handle ThreadLocal cleanup with TransmittableThreadLocal, understand queue capacity immutability, calculate thread counts using the WT/ST formula, and avoid parallelStream for I/O-heavy or stateful operations.