How to Determine the Optimal Number of Threads in a Java Thread Pool

To improve program processing speed, tasks are split into concurrent units and delegated to a thread pool, which reduces the overhead of creating and destroying threads, especially under high concurrency.

Two main approaches are presented for estimating the optimal number of threads.

Approach 1 (Java Concurrency in Practice) : For CPU‑bound tasks, use Ncpu + 1 threads; for I/O‑bound tasks, estimate the waiting‑to‑computing ratio (W/C) and calculate Nthreads = Ncpu × Ucpu × (1 + W/C) , where Ucpu is the desired CPU utilization (0‑1).

Definitions used:

Ncpu = number of CPU cores
Ucpu = target CPU utilization (0 ≤ Ucpu ≤ 1)
W/C = waiting time / computation time ratio
Nthreads = Ncpu × Ucpu × (1 + W/C)

You can obtain the CPU core count at runtime with:

int N_CPUS = Runtime.getRuntime().availableProcessors();

Other resources such as memory, file handles, sockets, and database connections also constrain pool size; the total required resources per task divided by the available amount gives an upper bound.

Approach 2 (Programming Concurrency on the JVM Mastering) : The minimum thread count should equal the number of available CPU cores. For I/O‑bound tasks, increase the pool size proportionally to the blocking coefficient, using the formula Nthreads = Ncpu / (1 - blockingCoefficient) . The blocking coefficient can be approximated as W / (W + C) .

Practical example: To achieve 20 TPS with each transaction taking 4 seconds, each thread processes 0.25 TPS, so 80 threads are needed. However, the actual limit is governed by CPU capacity, and the optimal pool size follows the formulas above.

Guidelines derived:

CPU‑bound (compute‑intensive) tasks: use roughly Ncpu (or Ncpu + 1 ) threads.

I/O‑bound tasks: use about 2 × Ncpu threads, adjusting based on testing.

Additional considerations include Amdahl's law for parallel speedup, the overhead of context switching, and cases where a single‑threaded design (e.g., Redis) can be more efficient for memory‑bound workloads.

In summary, choose the thread pool size based on the nature of the workload, CPU core count, and the estimated blocking ratio, using the presented formulas as a starting point.