Mastering Java Thread Pools: Design, Implementation, and Best Practices

1. What Is a Thread

A thread is the smallest unit of execution that the operating system can schedule; it lives inside a process and represents a single sequential flow of control. Multiple threads can run concurrently within the same process.

2. Creating Threads in Java

2.1 Creating Threads

/**
 * Extend Thread and override run()
 */
class MyThread extends Thread {
    @Override
    public void run() {
        System.out.println("myThread..." + Thread.currentThread().getName());
    }
}

/**
 * Implement Runnable
 */
class MyRunnable implements Runnable {
    @Override
    public void run() {
        System.out.println("MyRunnable..." + Thread.currentThread().getName());
    }
}

/**
 * Implement Callable with a return value
 */
class MyCallable implements Callable<String> {
    @Override
    public String call() throws Exception {
        return "MyCallable..." + Thread.currentThread().getName();
    }
}

2.2 Test

public static void main(String[] args) throws Exception {
    MyThread thread = new MyThread();
    thread.run();   // myThread...main
    thread.start(); // myThread...Thread-0

    MyRunnable myRunnable = new MyRunnable();
    Thread thread1 = new Thread(myRunnable);
    myRunnable.run();   // MyRunnable...main
    thread1.start();     // MyRunnable...Thread-1

    MyCallable myCallable = new MyCallable();
    FutureTask<String> futureTask = new FutureTask<>(myCallable);
    Thread thread2 = new Thread(futureTask);
    thread2.start();
    System.out.println(myCallable.call());          // MyCallable...main
    System.out.println(futureTask.get());           // MyCallable...Thread-2
}

2.3 Question

Why does directly calling run() behave differently from calling start()? Does new Thread() really create a thread?

2.4 Analysis

Calling run() executes the task in the current (main) thread, while start() creates a new OS‑level thread. The actual thread creation happens inside the native start0() method invoked by Thread.start().

2.5 Summary

Java does not execute tasks directly in a new thread; it creates a Thread object that delegates to the OS to start a thread.

Implementing Runnable separates the task from thread creation, decoupling start‑up from execution.

Implementing Callable with Future allows the task to return a result after completion.

3. Multithreading

3.1 What Is Multithreading

Multithreading is a technique that enables multiple threads to execute concurrently, allowing several tasks to be processed at the same time instead of sequentially.

3.2 Benefits of Multithreading

3.2.1 Serial Processing Example

public static void main(String[] args) throws Exception {
    System.out.println("start...");
    long start = System.currentTimeMillis();
    for (int i = 0; i < 5; i++) {
        Thread.sleep(2000); // each task takes 2 seconds
        System.out.println("task done...");
    }
    long end = System.currentTimeMillis();
    System.out.println("end...,time = " + (end - start));
}
// Output shows ~10 seconds total

3.2.2 Parallel Processing Example

public static void main(String[] args) throws Exception {
    System.out.println("start...");
    long start = System.currentTimeMillis();
    List<Future<String>> list = new ArrayList<>();
    for (int i = 0; i < 5; i++) {
        Callable<String> callable = () -> {
            Thread.sleep(2000);
            return "task done...";
        };
        FutureTask<String> task = new FutureTask<>(callable);
        list.add(task);
        new Thread(task).start();
    }
    for (Future<String> f : list) {
        System.out.println(f.get());
    }
    long end = System.currentTimeMillis();
    System.out.println("end...,time = " + (end - start));
}
// Output shows ~2 seconds total

3.3 Summary

Multithreading splits a job into several sub‑tasks that can run concurrently.

The goal is to improve resource utilization and overall system efficiency, not merely raw speed.

3.4 Thread‑Pool Design

To avoid the overhead of creating a large number of threads, a thread pool limits the maximum number of threads, reuses idle threads, queues excess tasks, and provides policies for task rejection.

3.4.1 Basic Functions

Limit the total number of threads to prevent resource exhaustion.

Reuse existing threads instead of creating new ones for each task.

3.4.2 Problems to Solve

How to reuse threads after they finish a task.

What to do when the number of pending tasks exceeds the pool size.

3.4.3 Analogy

Imagine a logistics company where tasks are cargo, trucks are threads, and the depot is the thread pool. The company buys a limited number of trucks, reuses them for new cargo, rents extra trucks during peak periods, and rejects cargo when capacity is exhausted.

3.5 Java Thread‑Pool Implementation

3.5.1 Design Overview

public class ThreadPoolExecutor extends AbstractExecutorService {
    private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
    private static final int COUNT_BITS = Integer.SIZE - 3;
    private static final int CAPACITY   = (1 << COUNT_BITS) - 1;
    private static final int RUNNING    = -1 << COUNT_BITS;
    private static final int SHUTDOWN   =  0 << COUNT_BITS;
    private static final int STOP       =  1 << COUNT_BITS;
    private static final int TIDYING    =  2 << COUNT_BITS;
    private static final int TERMINATED =  3 << COUNT_BITS;

    private final BlockingQueue<Runnable> workQueue;
    private final ReentrantLock mainLock = new ReentrantLock();
    private final HashSet<Worker> workers = new HashSet<>();
    private final Condition termination = mainLock.newCondition();
    private int largestPoolSize;
    private long completedTaskCount;
    private volatile ThreadFactory threadFactory;
    private volatile RejectedExecutionHandler handler;
    private volatile long keepAliveTime;
    private volatile boolean allowCoreThreadTimeOut;
    private volatile int corePoolSize;
    private volatile int maximumPoolSize;
    private static final RejectedExecutionHandler defaultHandler = new AbortPolicy();
    private static final RuntimePermission shutdownPerm = new RuntimePermission("modifyThread");
    ...
}

3.5.2 Constructor

public ThreadPoolExecutor(int corePoolSize,
                           int maximumPoolSize,
                           long keepAliveTime,
                           TimeUnit unit,
                           BlockingQueue<Runnable> workQueue,
                           ThreadFactory threadFactory,
                           RejectedExecutionHandler handler) {
    if (corePoolSize < 0 || maximumPoolSize <= 0 ||
        maximumPoolSize < corePoolSize || keepAliveTime < 0)
        throw new IllegalArgumentException();
    if (workQueue == null || threadFactory == null || handler == null)
        throw new NullPointerException();
    this.corePoolSize = corePoolSize;
    this.maximumPoolSize = maximumPoolSize;
    this.workQueue = workQueue;
    this.keepAliveTime = unit.toNanos(keepAliveTime);
    this.threadFactory = threadFactory;
    this.handler = handler;
}

3.5.3 Task Submission Methods

public Future<?> submit(Runnable task) {
    if (task == null) throw new NullPointerException();
    RunnableFuture<Void> ftask = newTaskFor(task, null);
    execute(ftask);
    return ftask;
}

public void execute(Runnable command) {
    if (command == null) throw new NullPointerException();
    int c = ctl.get();
    if (workerCountOf(c) < corePoolSize) {
        if (addWorker(command, true))
            return;
        c = ctl.get();
    }
    if (isRunning(c) && workQueue.offer(command)) {
        int recheck = ctl.get();
        if (!isRunning(recheck) && remove(command))
            reject(command);
        else if (workerCountOf(recheck) == 0)
            addWorker(null, false);
    } else if (!addWorker(command, false))
        reject(command);
}

3.5.4 addWorker

private boolean addWorker(Runnable firstTask, boolean core) {
    retry:
    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);
        if (rs >= SHUTDOWN &&
            !(rs == SHUTDOWN && firstTask == null && !workQueue.isEmpty()))
            return false;
        for (;;) {
            int wc = workerCountOf(c);
            if (wc >= CAPACITY || wc >= (core ? corePoolSize : maximumPoolSize))
                return false;
            if (compareAndIncrementWorkerCount(c))
                break retry;
            c = ctl.get();
            if (runStateOf(c) != rs)
                continue retry;
        }
    }
    boolean workerStarted = false;
    boolean workerAdded = false;
    Worker w = null;
    try {
        w = new Worker(firstTask);
        Thread t = w.thread;
        if (t != null) {
            final ReentrantLock mainLock = this.mainLock;
            mainLock.lock();
            try {
                int rs = runStateOf(ctl.get());
                if (rs < SHUTDOWN || (rs == SHUTDOWN && firstTask == null)) {
                    if (t.isAlive())
                        throw new IllegalThreadStateException();
                    workers.add(w);
                    int s = workers.size();
                    if (s > largestPoolSize) largestPoolSize = s;
                    workerAdded = true;
                }
            } finally {
                mainLock.unlock();
            }
            if (workerAdded) {
                t.start();
                workerStarted = true;
            }
        }
    } finally {
        if (!workerStarted)
            addWorkerFailed(w);
    }
    return workerStarted;
}

3.5.5 runWorker

final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    Runnable task = w.firstTask;
    w.firstTask = null;
    w.unlock();
    boolean completedAbruptly = true;
    try {
        while (task != null || (task = getTask()) != null) {
            w.lock();
            if ((runStateAtLeast(ctl.get(), STOP) ||
                 (Thread.interrupted() && runStateAtLeast(ctl.get(), STOP))) &&
                !wt.isInterrupted())
                wt.interrupt();
            try {
                beforeExecute(wt, task);
                Throwable thrown = null;
                try {
                    task.run();
                } catch (RuntimeException x) { thrown = x; throw x; }
                catch (Error x) { thrown = x; throw x; }
                catch (Throwable x) { thrown = x; throw new Error(x); }
                finally {
                    afterExecute(task, thrown);
                }
            } finally {
                task = null;
                w.completedTasks++;
                w.unlock();
            }
        }
        completedAbruptly = false;
    } finally {
        processWorkerExit(w, completedAbruptly);
    }
}

3.5.6 getTask

private Runnable getTask() {
    boolean timedOut = false;
    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);
        if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
            decrementWorkerCount();
            return null;
        }
        int wc = workerCountOf(c);
        boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
        if ((wc > maximumPoolSize || (timed && timedOut)) && (wc > 1 || workQueue.isEmpty())) {
            if (compareAndDecrementWorkerCount(c))
                return null;
            continue;
        }
        try {
            Runnable r = timed ?
                workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
                workQueue.take();
            if (r != null) return r;
            timedOut = true;
        } catch (InterruptedException retry) {
            timedOut = false;
        }
    }
}

3.5.7 processWorkerExit

private void processWorkerExit(Worker w, boolean completedAbruptly) {
    if (completedAbruptly) decrementWorkerCount();
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        completedTaskCount += w.completedTasks;
        workers.remove(w);
    } finally {
        mainLock.unlock();
    }
    tryTerminate();
    int c = ctl.get();
    if (runStateLessThan(c, STOP)) {
        if (!completedAbruptly) {
            int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
            if (min == 0 && !workQueue.isEmpty()) min = 1;
            if (workerCountOf(c) >= min) return;
        }
        addWorker(null, false);
    }
}

3.5.8 tryTerminate

final void tryTerminate() {
    for (;;) {
        int c = ctl.get();
        if (isRunning(c) ||
            runStateAtLeast(c, TIDYING) ||
            (runStateOf(c) == SHUTDOWN && !workQueue.isEmpty()))
            return;
        if (workerCountOf(c) != 0) {
            interruptIdleWorkers(ONLY_ONE);
            return;
        }
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
                try {
                    terminated();
                } finally {
                    ctl.set(ctlOf(TERMINATED, 0));
                    termination.signalAll();
                }
                return;
            }
        } finally {
            mainLock.unlock();
        }
    }
}

3.6 Summary of Java Thread Pool

Manages thread count and lifetimes.

Provides a task queue.

Implements thread reuse.

Handles task overload with rejection policies.

Offers statistics, exception handling, and customizable hooks.

3.7 Usage Example

public static void main(String[] args) throws Exception {
    ThreadPoolExecutor pool = new ThreadPoolExecutor(
            5, 10, 100, TimeUnit.SECONDS, new ArrayBlockingQueue<>(5));

    // Submit 4 tasks – only core threads are created
    for (int i = 0; i < 4; i++) {
        pool.submit(new MyRunnable());
    }
    System.out.println("worker count = " + pool.getPoolSize()); // 4
    System.out.println("queue size = " + pool.getQueue().size()); // 0

    // Submit another 4 tasks – fills core pool and adds to queue
    for (int i = 0; i < 4; i++) {
        pool.submit(new MyRunnable());
    }
    System.out.println("worker count = " + pool.getPoolSize()); // 5
    System.out.println("queue size = " + pool.getQueue().size()); // 3

    // Submit 4 more – queue full, creates non‑core threads
    for (int i = 0; i < 4; i++) {
        pool.submit(new MyRunnable());
    }
    System.out.println("worker count = " + pool.getPoolSize()); // 7
    System.out.println("queue size = " + pool.getQueue().size()); // 5

    // Submit final 4 – excess tasks are rejected
    for (int i = 0; i < 4; i++) {
        try {
            pool.submit(new MyRunnable());
        } catch (Exception e) {
            e.printStackTrace(); // RejectedExecutionException
        }
    }
    System.out.println("worker count = " + pool.getPoolSize()); // 10
    System.out.println("queue size = " + pool.getQueue().size()); // 5
    System.out.println(pool.getTaskCount()); // 15 tasks executed

    Thread.sleep(1500);
    System.out.println("worker count = " + pool.getPoolSize()); // core threads remain
    System.out.println("queue size = " + pool.getQueue().size()); // 0

    pool.shutdown();
}