How to Size Java Thread Pools for Optimal Performance

Creating threads in Java incurs significant overhead, increasing request latency and consuming JVM and OS resources. Thread pools mitigate this cost by reusing a fixed set of threads.

1 Why Use a Thread Pool

Performance : Reduces the cost of repeatedly creating and destroying threads.

Scalability : Allows the pool to grow to meet load while limiting resource usage.

Resource Management : Caps the number of concurrent threads, preventing memory exhaustion.

2 Determining Pool Size – System and Resource Limits

Understanding hardware limits (CPU cores, memory) and external dependencies (database connections, third‑party services) is crucial. The article uses a web service handling HTTP requests that may query a database and call external APIs as a running example.

2.1 Scenario

For a web application processing incoming HTTP requests, each request may involve database access (via HikariCP) and external service calls. The goal is to choose a thread‑pool size that handles the load efficiently.

2.2 Factors to Consider

Database connection pool : If HikariCP is configured for a maximum of 100 connections, creating more threads than connections leads to waiting and contention.

import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;

public class DatabaseConnectionExample {
    public static void main(String[] args) {
        HikariConfig config = new HikariConfig();
        config.setJdbcUrl("jdbc:mysql://localhost:3306/mydb");
        config.setUsername("username");
        config.setPassword("password");
        config.setMaximumPoolSize(100); // max connections
        HikariDataSource dataSource = new HikariDataSource(config);
        // use dataSource to obtain connections
    }
}

External service throughput : The service can only handle a limited number of concurrent requests (e.g., 10). Over‑submitting requests can overload the service.

CPU cores : The number of available processors determines the upper bound for CPU‑bound work.

int numOfCores = Runtime.getRuntime().availableProcessors();

Running more threads than cores can cause excessive context switching, degrading performance.

3 CPU‑Intensive vs I/O‑Intensive Tasks

CPU‑intensive tasks (e.g., video encoding, compilation, simulations, machine‑learning training) benefit from a pool size close to the number of CPU cores.

Parallel processing using ExecutorService and Executors.newFixedThreadPool can split work across cores.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class ParallelSquareCalculator {
    public static void main(String[] args) {
        int[] numbers = {1,2,3,4,5,6,7,8,9,10};
        int numThreads = Runtime.getRuntime().availableProcessors();
        ExecutorService executor = Executors.newFixedThreadPool(numThreads);
        for (int number : numbers) {
            executor.submit(() -> {
                int square = calculateSquare(number);
                System.out.println("Square of " + number + " is " + square);
            });
        }
        executor.shutdown();
        try { executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS); }
        catch (InterruptedException e) { Thread.currentThread().interrupt(); }
    }
    private static int calculateSquare(int number) {
        try { Thread.sleep(1000); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
        return number * number;
    }
}

I/O‑intensive tasks (file read/write, network calls, database queries) benefit from a larger pool that can keep I/O devices busy without overwhelming them.

Cache frequently accessed data, balance load, use SSDs, and choose efficient data structures to reduce I/O latency.

4 Choosing the Number of Threads

4.1 CPU‑Bound Tasks

Use the number of available cores, possibly leaving one core free for the OS.

int availableCores = Runtime.getRuntime().availableProcessors();
int threads = Math.max(availableCores - 1, 1);
ExecutorService pool = Executors.newFixedThreadPool(threads);

4.2 I/O‑Bound Tasks

Base the pool size on expected I/O latency and desired concurrency. A smaller pool can still achieve high throughput if I/O operations dominate.

int expectedIOLatency = 500; // ms
int threads = 4; // tuned based on latency and system capacity
ExecutorService pool = Executors.newFixedThreadPool(threads);

5 A Practical Formula

Number of threads =

Available Cores × Target CPU Utilization × (1 + WaitTime / ServiceTime)

Where:

Available Cores : Physical or logical CPU cores.

Target CPU Utilization : Desired percentage of CPU usage (e.g., 0.5 for 50%).

WaitTime : Average time a thread spends waiting for I/O.

ServiceTime : Average time spent doing computation.

6 Example Calculation

Server with 4 cores, aiming for 50% CPU utilization.

I/O‑bound task, blocking coefficient 0.5: 4 × 0.5 × (1 + 0.5) = 3 threads.

CPU‑bound task, blocking coefficient 0.1: 4 × 0.5 × (1 + 0.1) = 2.2 ≈ 2 threads.

Result: create two separate pools—one with 3 threads for I/O work, another with 2 threads for CPU work.