Optimizing Thread Pool Size for CPU‑Bound and I/O‑Bound Tasks in Java

CPU‑bound tasks require a lot of processing power, such as audio/video encoding, software compilation, complex simulations, machine‑learning or data‑mining jobs, and gaming.

Audio or video encoding/decoding

Compiling and linking software

Running complex simulations

Executing machine‑learning or data‑mining tasks

Playing video games

Optimization for CPU‑bound tasks

Multithreading and parallelism: Split a large job into smaller sub‑tasks and distribute them across multiple CPU cores to increase overall performance.

Example: calculate the square of each number in a large array using several threads.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class ParallelSquareCalculator {
    public static void main(String[] args) {
        int[] numbers = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
        int numThreads = Runtime.getRuntime().availableProcessors(); // Get the number of CPU cores
        ExecutorService executorService = Executors.newFixedThreadPool(numThreads);

        for (int number : numbers) {
            executorService.submit(() -> {
                int square = calculateSquare(number);
                System.out.println("Square of " + number + " is " + square);
            });
        }

        executorService.shutdown();
        try {
            executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }

    private static int calculateSquare(int number) {
        // Simulate a time‑consuming calculation (e.g., database query, complex computation)
        try {
            Thread.sleep(1000); // Simulate a 1‑second delay
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        return number * number;
    }
}

IO‑bound tasks

IO‑bound tasks spend most of their time waiting for storage devices, network sockets, or user input.

Reading or writing large files (e.g., video files, database loads)

Downloading or uploading files over the network

Sending and receiving email

Running network servers or other network services

Executing database queries

Handling incoming requests in a web server

Optimization for IO‑bound tasks

Caching: Store frequently accessed data in memory to reduce repeated IO operations.

Load balancing: Distribute IO‑bound work across multiple threads or processes.

Use SSDs: Solid‑state drives dramatically speed up IO compared with HDDs.

Efficient data structures: Hash tables, B‑trees, etc., reduce the number of IO operations needed.

Avoid unnecessary file operations: Reduce repeated open/close cycles.

Determine thread count

4 For CPU‑bound tasks

Use a thread count close to the number of available CPU cores; too many threads cause excessive context switching.

Real‑world example: video encoding

Video encoding is CPU‑intensive; a multi‑core CPU can run several encoding threads in parallel.

Determine CPU‑bound thread count

In Java, Runtime.getRuntime().availableProcessors() returns the core count (e.g., 8 cores). Create a thread pool slightly smaller than the core count to leave capacity for the OS.

Create thread pool: Use a pool size of 6‑7 for an 8‑core machine.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class VideoEncodingApp {
    public static void main(String[] args) {
        int availableCores = Runtime.getRuntime().availableProcessors();
        int numberOfThreads = Math.max(availableCores - 1, 1); // Adjust as needed

        ExecutorService threadPool = Executors.newFixedThreadPool(numberOfThreads);

        // Submit video encoding tasks to the thread pool.
        for (int i = 0; i < 10; i++) {
            threadPool.execute(() -> {
                encodeVideo(); // Simulated video encoding task
            });
        }

        threadPool.shutdown();
    }

    private static void encodeVideo() {
        // Simulate video encoding (CPU‑bound) task.
        // Complex calculations and compression algorithms here.
    }
}

5 For I/O‑bound tasks

The optimal thread count depends on the expected I/O latency rather than the number of CPU cores; the goal is to keep I/O devices busy without overloading them.

Real‑world example: web crawling

A crawler issues HTTP requests; network latency makes each request I/O‑bound.

Determine I/O‑bound thread count

Estimate the average I/O latency (e.g., 500 ms per request) and choose a pool size that allows enough overlap.

Create thread pool: A modest pool (e.g., 4 threads) can efficiently handle many concurrent HTTP calls.

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class WebPageCrawler {
    public static void main(String[] args) {
        int expectedIOLatency = 500; // Estimated I/O latency in milliseconds
        int numberOfThreads = 4; // Adjust based on your expected latency and system capabilities

        ExecutorService threadPool = Executors.newFixedThreadPool(numberOfThreads);

        // List of URLs to crawl.
        String[] urlsToCrawl = {
            "https://example.com",
            "https://google.com",
            "https://github.com",
            // Add more URLs here
        };

        for (String url : urlsToCrawl) {
            threadPool.execute(() -> {
                crawlWebPage(url, expectedIOLatency);
            });
        }

        threadPool.shutdown();
    }

    private static void crawlWebPage(String url, int expectedIOLatency) {
        // Simulate web page crawling (I/O‑bound) task.
        // Perform HTTP request and process the page content.
        try {
            Thread.sleep(expectedIOLatency); // Simulating I/O latency
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
}

6 Can we follow a concrete formula?

Thread count can be estimated with:

threads = cores × target CPU utilization × (1 + wait time / service time)

Where:

cores: Number of CPU cores available to the application.

target CPU utilization: Desired percentage of CPU time (e.g., 0.5 for 50%).

wait time: Time a thread spends waiting for I/O.

service time: Time spent doing actual computation.

blocking coefficient: Ratio of wait time to service time.

7 Usage example

Assume a 4‑core server with a goal of using 50 % of CPU capacity.

For I/O‑bound tasks with a blocking coefficient of 0.5:

threads = 4 cores × 0.5 × (1 + 0.5) = 3 threads

For CPU‑bound tasks with a blocking coefficient of 0.1:

threads = 4 cores × 0.5 × (1 + 0.1) = 2.2 ≈ 2 threads

Thus, create two thread pools: one with 3 threads for I/O‑bound work and another with 2 threads for CPU‑bound work. This formula provides a practical starting point that can be tuned to real‑world conditions.

-End-