How Many Threads Should You Run? Real‑World Tests Reveal the Truth

Thread Count and CPU Utilization Experiments

Many people follow a simple rule of thumb for thread numbers: CPU‑intensive programs use cores + 1 threads, while I/O‑intensive programs use cores × 2 . The author questions this theory and runs a series of tests.

Test Setup

Environment: AMD Ryzen 5 3600, 6 physical cores, 12 logical threads.

CPU‑Bound Single Thread

public class CPUUtilizationTest {
    public static void main(String[] args) {
        // dead loop, does nothing
        while (true) {
        }
    }
}

The single thread fully occupies one core (core 3 reaches 100% utilization).

Increasing Thread Count (CPU‑Bound)

public class CPUUtilizationTest {
    public static void main(String[] args) {
        for (int j = 0; j < 6; j++) {
            new Thread(new Runnable() {
                @Override
                public void run() {
                    while (true) {
                    }
                }
            }).start();
        }
    }
}

With 6 threads, six cores become fully utilized. Adding up to 12 threads fills all cores, but further threads do not increase utilization; they only raise the load average.

I/O‑Bound Scenario

public class CPUUtilizationTest {
    public static void main(String[] args) throws InterruptedException {
        new Thread(new Runnable() {
            @Override
            public void run() {
                while (true) {
                    for (int i = 0; i < 100_000_000L; i++) {
                    }
                    Thread.sleep(50); // simulate I/O wait
                }
            }
        }).start();
    }
}

With one thread performing periodic sleep, the utilized core drops to about 50%. Increasing to 12 threads raises each core’s utilization to ~60%. With 18 threads it approaches 100%, demonstrating that I/O‑wait periods let the OS schedule other threads, improving overall CPU usage.

When the I/O wait frequency is increased (shorter compute loop, same 18 threads), per‑core utilization falls to around 70%, showing that more frequent I/O pauses reduce CPU load per core.

Key Takeaways

One CPU‑bound thread can fully occupy a core; a multi‑core CPU can run at most as many such threads as it has cores.

Running more CPU‑bound threads than cores causes excessive context switching, raising load without improving throughput.

I/O‑bound threads free the CPU during wait periods, allowing the OS to keep the CPU busy with other work and increasing overall utilization.

The higher the I/O‑wait frequency or the longer the wait, the lower the per‑core utilization, but more threads can be scheduled concurrently.

Thread‑Count Formula (from *Java Concurrency in Practice*)

To achieve a target CPU utilization:

threads = cores × targetUtil × (1 + waitTime / computeTime)

Example: target 90% utilization on 12 cores with a 50 ms wait and a 50 ms compute loop yields ≈ 22 threads.

Conversely, you can estimate utilization from thread count:

utilization ≈ threads / (cores × (1 + waitTime / computeTime))

In real applications, accurately measuring wait and compute times is difficult because workloads mix computation, memory access, and I/O.

Practical Thread‑Pool Sizing

In production Java services (e.g., Spring Boot with Tomcat, HikariCP, G1 GC), many other threads already exist. Therefore, the formula alone is insufficient; you must:

Identify other processes and JVM threads that may compete for CPU.

Define goals such as acceptable CPU utilization, GC pause frequency, and throughput.

Analyze bottlenecks (e.g., external service limits, connection‑pool size).

Iteratively increase or decrease thread‑pool size and measure the impact.

There is no universal answer; start with the number of CPU cores and adjust based on testing and the specific workload.

Appendix

Java: Get CPU Core Count

Runtime.getRuntime().availableProcessors(); // returns logical core count, e.g., 12 for a 6‑core/12‑thread CPU

Linux: Get CPU Core Count

# Total physical cores = physical CPUs × cores per CPU
cat /proc/cpuinfo | grep "physical id" | sort | uniq | wc -l

# Cores per physical CPU
cat /proc/cpuinfo | grep "cpu cores" | uniq

# Logical CPUs
cat /proc/cpuinfo | grep "processor" | wc -l

Images illustrating the test results are omitted for brevity.