What Happens to a Java ThreadPoolExecutor When an OOM Occurs?

ThreadPoolExecutor Behavior When OOM Occurs

Environment: JDK 1.8. The example creates a fixed‑size ThreadPoolExecutor with three worker threads and a small queue, then forces an OutOfMemoryError by allocating large byte arrays in one task while limiting the JVM heap to -Xms10m -Xmx10m.

public static void main(String[] args) {
    ThreadPoolExecutor pool = new ThreadPoolExecutor(
        3, 3, 60, TimeUnit.SECONDS,
        new ArrayBlockingQueue<Runnable>(2));
    // three tasks: two simple counters, one that allocates memory
    pool.execute(() -> {
        int i = 0;
        while (true) {
            System.out.println(Thread.currentThread().getName() + ", i = " + (i++) + "," + pool);
            TimeUnit.MILLISECONDS.sleep(50);
        }
    });
    pool.execute(() -> {
        int j = 0;
        while (true) {
            System.out.println(Thread.currentThread().getName() + ", j = " + (j++) + "," + pool);
            TimeUnit.MILLISECONDS.sleep(50);
        }
    });
    pool.execute(() -> {
        int k = 0;
        List<byte[]> datas = new ArrayList<>();
        while (true) {
            System.out.println(Thread.currentThread().getName() + ", k = " + (k++) + "," + pool);
            byte[] buf = new byte[1024 * 100];
            datas.add(buf);
            TimeUnit.MILLISECONDS.sleep(20);
        }
    });
}

When the JVM is started with -Xms10m -Xmx10m, the memory‑intensive task quickly throws an OutOfMemoryError. The pool stops the thread that caused the OOM, but the other two threads continue to run.

Custom Thread Exception Handling

ThreadPoolExecutor pool = new ThreadPoolExecutor(
    3, 3, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<Runnable>(2),
    new ThreadFactory() {
        private final ThreadGroup group = new ThreadGroup("Pack-Group");
        private final AtomicInteger threadNumber = new AtomicInteger(1);
        private final String namePrefix = "pool-custom-thread-";

        @Override
        public Thread newThread(Runnable r) {
            Thread t = new Thread(group, r, namePrefix + threadNumber.getAndIncrement(), 0);
            t.setUncaughtExceptionHandler((thread, e) -> {
                System.out.println("Custom thread exception handling: " + t.getName());
                e.printStackTrace();
            });
            return t;
        }
    });

ThreadPoolExecutor Rejection Policies

Four built‑in policies are demonstrated.

AbortPolicy

ThreadPoolExecutor pool = new ThreadPoolExecutor(
    3, 3, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<Runnable>(2),
    new ThreadPoolExecutor.AbortPolicy());

When submitted tasks exceed corePoolSize + queueCapacity, the extra tasks are rejected and a RejectedExecutionException is thrown.

CallerRunsPolicy

ThreadPoolExecutor pool = new ThreadPoolExecutor(
    3, 3, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<Runnable>(2),
    new ThreadPoolExecutor.CallerRunsPolicy());

The calling thread (the thread that invoked execute) runs the rejected task.

DiscardOldestPolicy

ThreadPoolExecutor pool = new ThreadPoolExecutor(
    3, 3, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<Runnable>(2),
    new ThreadPoolExecutor.DiscardOldestPolicy());

The oldest task in the queue is discarded, and the new task is inserted at the tail.

DiscardPolicy

ThreadPoolExecutor pool = new ThreadPoolExecutor(
    3, 3, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<Runnable>(2),
    new ThreadPoolExecutor.DiscardPolicy());

Rejected tasks are silently dropped; the pool continues processing remaining tasks.

Custom ThreadPool with Execution Timing

public class CustomThreadPool extends ThreadPoolExecutor {
    public CustomThreadPool(int corePoolSize, int maximumPoolSize,
                            long keepAliveTime, TimeUnit unit,
                            BlockingQueue<Runnable> workQueue) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue);
    }

    @Override
    protected void beforeExecute(Thread t, Runnable r) {
        super.beforeExecute(t, r);
        if (r instanceof Task) {
            ((Task) r).setStart(System.currentTimeMillis());
            System.out.println(t.getName() + ", 开始执行");
        }
    }

    @Override
    protected void afterExecute(Runnable r, Throwable t) {
        super.afterExecute(r, t);
        if (r instanceof Task) {
            ((Task) r).times();
        }
    }

    public void execute(Task command) {
        super.execute(command);
    }

    public static class Task implements Runnable {
        private long start;
        private Callback callback;
        public Task(Callback callback) { this.callback = callback; }
        @Override
        public void run() {
            if (callback != null) { callback.callback(); }
        }
        public void times() {
            System.out.println(Thread.currentThread().getName() + " 执行耗时：" + (System.currentTimeMillis() - start) + "ms");
        }
        public void setStart(long start) { this.start = start; }
    }

    public interface Callback { void callback(); }
}

By overriding beforeExecute and afterExecute, the custom pool logs the start time and prints the execution duration for each task.