Deep Dive into the Implementation of Java's ThreadPoolExecutor

1. Overview

A thread pool is a container that holds a large number of reusable threads, reducing the overhead of creating and destroying threads for each task.

ThreadPoolExecutor is the core class of the thread‑pool framework; the following analysis is based on JDK 1.8 source code.

2. ThreadPoolExecutor Attributes

The class defines several fields that control the pool state and worker count, using a single AtomicInteger ctl where the high 3 bits represent the run state and the low 29 bits represent the worker count.

private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
private static final int COUNT_BITS = Integer.SIZE - 3;
private static final int CAPACITY   = (1 << COUNT_BITS) - 1;
private static final int RUNNING    = -1 << COUNT_BITS;
private static final int SHUTDOWN   =  0 << COUNT_BITS;
private static final int STOP       =  1 << COUNT_BITS;
private static final int TIDYING    =  2 << COUNT_BITS;
private static final int TERMINATED =  3 << COUNT_BITS;
private final BlockingQueue<Runnable> workQueue;
private final ReentrantLock mainLock = new ReentrantLock();
private final HashSet<Worker> workers = new HashSet<>();
private final Condition termination = mainLock.newCondition();
private int largestPoolSize;
private long completedTaskCount;
private volatile ThreadFactory threadFactory;
private volatile RejectedExecutionHandler handler;
private volatile long keepAliveTime;
private volatile boolean allowCoreThreadTimeOut;
private volatile int corePoolSize;
private volatile int maximumPoolSize;
private static final RejectedExecutionHandler defaultHandler = new AbortPolicy();
private static final RuntimePermission shutdownPerm = new RuntimePermission("modifyThread");
private final AccessControlContext acc;

The pool state is managed through helper methods that extract the run state and worker count from ctl:

private static int runStateOf(int c) { return c & ~CAPACITY; }
private static int workerCountOf(int c) { return c & CAPACITY; }
private static int ctlOf(int rs, int wc) { return rs | wc; }
private static boolean isRunning(int c) { return c < SHUTDOWN; }

The possible state transitions are:

RUNNING → SHUTDOWN (via shutdown())

(RUNNING or SHUTDOWN) → STOP (via shutdownNow())

SHUTDOWN → TIDYING (when both queue and workers are empty)

STOP → TIDYING (when workers are empty)

TIDYING → TERMINATED (after terminated() completes)

3. Constructors

ThreadPoolExecutor provides several overloaded constructors that ultimately delegate to the most detailed one, which validates parameters and initializes all fields.

public ThreadPoolExecutor(int corePoolSize,
                          int maximumPoolSize,
                          long keepAliveTime,
                          TimeUnit unit,
                          BlockingQueue<Runnable> workQueue,
                          ThreadFactory threadFactory,
                          RejectedExecutionHandler handler) {
    if (corePoolSize < 0 || maximumPoolSize <= 0 ||
        maximumPoolSize < corePoolSize || keepAliveTime < 0)
        throw new IllegalArgumentException();
    if (workQueue == null || threadFactory == null || handler == null)
        throw new NullPointerException();
    this.acc = (System.getSecurityManager() == null) ? null : AccessController.getContext();
    this.corePoolSize = corePoolSize;
    this.maximumPoolSize = maximumPoolSize;
    this.workQueue = workQueue;
    this.keepAliveTime = unit.toNanos(keepAliveTime);
    this.threadFactory = threadFactory;
    this.handler = handler;
}

The parameters have the following meanings:

corePoolSize : number of core threads that are kept alive even when idle.

maximumPoolSize : upper bound of total threads; new threads are created when the queue is full.

keepAliveTime and unit : idle time after which non‑core threads are terminated.

workQueue : task queue (e.g., SynchronousQueue, LinkedBlockingQueue, ArrayBlockingQueue).

threadFactory : factory that creates new threads.

handler : rejection policy ( AbortPolicy, CallerRunsPolicy, DiscardPolicy, DiscardOldestPolicy).

4. ThreadPoolExecutor Core Logic

4.1 Submitting Tasks

Two submission methods exist: execute(Runnable) and submit(...). The submit overloads ultimately call execute after wrapping the task in a FutureTask.

public Future<?> submit(Runnable task) {
    if (task == null) throw new NullPointerException();
    RunnableFuture<Void> ftask = newTaskFor(task, null);
    execute(ftask);
    return ftask;
}

The execute method follows four steps:

If the current worker count is less than corePoolSize, try to add a core worker.

If the pool is running, enqueue the task in workQueue.

If enqueuing fails and the worker count is below maximumPoolSize, try to add a non‑core worker.

If all attempts fail, apply the rejection handler.

4.2 Creating Threads (addWorker)

The private method addWorker(Runnable firstTask, boolean core) performs the following:

Checks pool state and worker limits.

Atomically increments the worker count.

Creates a Worker (which implements Runnable and extends AbstractQueuedSynchronizer), obtains a thread from the ThreadFactory, and adds the worker to the workers set under mainLock.

Starts the thread.

If thread start fails, rolls back the worker creation.

private boolean addWorker(Runnable firstTask, boolean core) {
    retry:
    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);
        if (rs >= SHUTDOWN && !(rs == SHUTDOWN && firstTask == null && !workQueue.isEmpty()))
            return false;
        for (;;) {
            int wc = workerCountOf(c);
            if (wc >= CAPACITY || wc >= (core ? corePoolSize : maximumPoolSize))
                return false;
            if (compareAndIncrementWorkerCount(c))
                break retry;
            c = ctl.get(); // Recheck on failure
            if (runStateOf(c) != rs)
                continue retry;
        }
    }
    // Create and start worker ...
    return workerStarted;
}

4.3 Worker Implementation

The inner class Worker wraps a thread and the first task, and provides a non‑reentrant lock via AQS.

private final class Worker extends AbstractQueuedSynchronizer implements Runnable {
    final Thread thread;
    Runnable firstTask;
    volatile long completedTasks;
    Worker(Runnable firstTask) {
        setState(-1); // prevent interrupts until started
        this.firstTask = firstTask;
        this.thread = getThreadFactory().newThread(this);
    }
    public void run() { runWorker(this); }
    // lock/unlock implementations omitted for brevity
}

4.4 Worker Run Loop (runWorker)

Each worker repeatedly obtains tasks via getTask(), executes them while holding its lock, and invokes the optional beforeExecute and afterExecute hooks.

final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    Runnable task = w.firstTask;
    w.firstTask = null;
    w.unlock(); // allow interrupts
    boolean completedAbruptly = true;
    try {
        while (task != null || (task = getTask()) != null) {
            w.lock();
            if (runStateAtLeast(ctl.get(), STOP) ||
                (Thread.interrupted() && runStateAtLeast(ctl.get(), STOP)))
                wt.interrupt();
            try {
                beforeExecute(wt, task);
                Throwable thrown = null;
                try { task.run(); }
                catch (RuntimeException x) { thrown = x; throw x; }
                catch (Error x) { thrown = x; throw x; }
                catch (Throwable x) { thrown = x; throw new Error(x); }
                finally { afterExecute(task, thrown); }
            } finally {
                task = null;
                w.completedTasks++;
                w.unlock();
            }
        }
        completedAbruptly = false;
    } finally {
        processWorkerExit(w, completedAbruptly);
    }
}

4.5 Obtaining Tasks (getTask)

getTask()

retrieves work from the queue, respecting timeout and pool state. It returns null when the pool should shrink or terminate.

private Runnable getTask() {
    boolean timedOut = false;
    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);
        if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
            decrementWorkerCount();
            return null;
        }
        int wc = workerCountOf(c);
        boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
        if ((wc > maximumPoolSize || (timed && timedOut)) && (wc > 1 || workQueue.isEmpty())) {
            if (compareAndDecrementWorkerCount(c))
                return null;
            continue;
        }
        try {
            Runnable r = timed ?
                workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
                workQueue.take();
            if (r != null) return r;
            timedOut = true;
        } catch (InterruptedException retry) {
            timedOut = false;
        }
    }
}

4.6 Worker Exit Processing (processWorkerExit)

When a worker terminates, this method updates completed task counts, removes the worker from the set, and attempts pool termination.

private void processWorkerExit(Worker w, boolean completedAbruptly) {
    if (completedAbruptly) decrementWorkerCount();
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        completedTaskCount += w.completedTasks;
        workers.remove(w);
    } finally { mainLock.unlock(); }
    tryTerminate();
    int c = ctl.get();
    if (runStateLessThan(c, STOP)) {
        if (!completedAbruptly) {
            int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
            if (min == 0 && !workQueue.isEmpty()) min = 1;
            if (workerCountOf(c) >= min) return;
        }
        addWorker(null, false);
    }
}

4.7 Pool Termination (tryTerminate)

The method transitions the pool through TIDYING to TERMINATED when no workers remain and the queue is empty.

final void tryTerminate() {
    for (;;) {
        int c = ctl.get();
        if (isRunning(c) || runStateAtLeast(c, TIDYING) ||
            (runStateOf(c) == SHUTDOWN && !workQueue.isEmpty()))
            return;
        if (workerCountOf(c) != 0) {
            interruptIdleWorkers(ONLY_ONE);
            return;
        }
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
                try { terminated(); }
                finally {
                    ctl.set(ctlOf(TERMINATED, 0));
                    termination.signalAll();
                }
                return;
            }
        } finally { mainLock.unlock(); }
    }
}

5. Shutting Down the Pool

5.1 shutdown()

Transitions the pool to SHUTDOWN, stops accepting new tasks, interrupts idle workers, and invokes onShutdown().

public void shutdown() {
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        checkShutdownAccess();
        advanceRunState(SHUTDOWN);
        interruptIdleWorkers();
        onShutdown();
    } finally { mainLock.unlock(); }
    tryTerminate();
}

5.2 shutdownNow()

Sets the state to STOP, interrupts all workers, drains the queue, and returns the list of pending tasks.

public List<Runnable> shutdownNow() {
    List<Runnable> tasks;
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        checkShutdownAccess();
        advanceRunState(STOP);
        interruptWorkers();
        tasks = drainQueue();
    } finally { mainLock.unlock(); }
    tryTerminate();
    return tasks;
}

6. Conclusion

The ThreadPoolExecutor class demonstrates sophisticated design: packing run state and worker count into a single atomic integer, using CAS for lock‑free updates, employing an inner Worker class that extends AQS to provide a custom non‑reentrant lock, and implementing a robust thread‑reuse loop that efficiently processes tasks while handling shutdown and termination semantics.

Understanding these mechanisms gives developers deep insight into Java's concurrency utilities and helps in tuning thread‑pool parameters for real‑world backend systems.