When and How to Use Multithreading: Choosing the Right Thread Count

Why Use Multithreading?

The best way to avoid concurrency bugs is not to write concurrent code.

Even though multithreaded programming is error‑prone, it persists because it can dramatically improve performance in the right situations.

What Scenarios Suit Concurrent Programming?

Two main categories determine whether concurrency is beneficial:

CPU‑intensive programs

I/O‑intensive programs

Key questions to ask:

Is concurrent programming fast in all scenarios?

What does “fast” mean and how do we measure it?

Using multithreading means setting the correct number of threads for the right scenario to maximize program speed.

At the hardware level, this translates to fully utilizing CPU and I/O resources.

CPU‑Intensive Programs

A complete request where I/O finishes quickly leaves most work for the CPU, so the CPU computation proportion is large.

Example: summing numbers from 1 to 10 billion. On a single‑core CPU, dividing the work into four threads (each handling a range) does not improve speed because all threads share the same CPU time slice and incur context‑switch overhead.

Thread 1 computes [1, 2.5 billion) …

Thread 4 computes [7.5 billion, 10 billion] Result: on a single core, total execution time is roughly the same as a single thread, and the overhead makes multithreading unsuitable.

On a multi‑core CPU (e.g., 4 cores), creating four threads allows each thread to run on its own core, eliminating waiting for time slices and reducing context‑switch overhead, theoretically achieving a four‑fold speedup.

Thread‑wait time proportion high → need more threads; thread‑CPU‑time proportion high → need fewer threads.

I/O‑Intensive Programs

A complete request where, after CPU work, a large amount of I/O remains, so I/O dominates the execution time.

During I/O operations the CPU is idle; therefore, we should create additional threads to keep the CPU busy while I/O is in progress.

When I/O time is three times CPU time, adding a fourth thread can fully utilize the CPU.

Optimal thread count = (1 / CPU utilization) = 1 + (I/O time / CPU time).

For a system with N CPU cores, the optimal number of threads for an I/O‑bound program is:

Optimal threads = N * (1 + (I/O time / CPU time)).

If I/O dominates (CPU utilization near 0), a practical rule of thumb is roughly 2 N (or 2 N + 1) threads.

How Many Threads Are Appropriate?

Interviewers often ask candidates to calculate the proper thread count.

CPU‑Intensive Programs

Theoretical rule: thread count = logical CPU cores. In practice, use logical CPU cores + 1 to account for occasional pauses (e.g., page faults) that leave a core idle.

From *Java Concurrency in Practice*: an extra thread ensures the CPU stays busy when a thread is temporarily blocked.

I/O‑Intensive Programs

Formula:

optimal threads = (1 / CPU utilization) = 1 + (I/O time / CPU time)

. Multiply by the number of CPU cores for multi‑core systems.

Example: with 8 CPU cores, CPU utilization 5 ms and I/O 100 ms, optimal threads = 8 * (1 + 100/5) = 168.

APM tools such as SkyWalking, CAT, and Zipkin can measure CPU utilization and I/O time to refine this estimate.

Will Adding More CPU Cores Always Solve the Problem?

Increasing core count helps only when the workload is truly CPU‑bound. For I/O‑bound or lock‑contention scenarios, adding cores may not improve performance and can even worsen it due to increased contention.

Minimize the size of critical sections; they are the bottleneck in concurrent programs.

Summary

Multithreading is advantageous when a program has a significant I/O or waiting component; pure CPU‑bound tasks may not benefit on a single core. Use qualitative analysis to estimate thread count, then refine with quantitative data from profiling tools. Finally, consider thread pools to manage thread lifecycle efficiently.