How to Design a ThreadPool to Send 10 Million SMS in One Hour – Interview Insights

Introduction

A candidate was asked in an interview to design a thread pool that could send 10 million marketing SMS messages within one hour. The interviewer probed the candidate on core pool size, maximum size, and rejection policy, exposing common pitfalls.

Why Executors are a “no‑go” in production

Using Executors.newFixedThreadPool creates a LinkedBlockingQueue with Integer.MAX_VALUE capacity, which can cause OOM when processing tens of millions of tasks. newCachedThreadPool can create an unbounded number of threads, leading to CPU saturation.

Three practical levels of thread‑pool design

Level 1 – “golden formula” for initial sizing

First determine whether the workload is CPU‑bound or I/O‑bound; SMS sending is I/O‑bound. Apply the formula: core = CPU cores × target CPU utilization × (wait time ÷ compute time) . For massive I/O tasks start with a modest core count and adjust after load testing.

Level 2 – Dynamic tuning & full‑stack monitoring

Do not hard‑code coreSize, maxSize, or queueSize. Retrieve them from a configuration centre (e.g., Apollo, Nacos). Continuously monitor queue remaining capacity and pool activity; trigger alerts or auto‑scale when queue usage exceeds 80%.

Level 3 – Rejection policy as the ultimate safeguard

When the pool is saturated, choose CallerRunsPolicy (recommended) instead of the default AbortPolicy. This provides natural back‑pressure: the submitting thread (often the main thread) processes the task, slowing down further task production and preventing OOM.

In offline batch scenarios this back‑pressure becomes a powerful technique, whereas in online web services it would block request threads.

Reliability measures for crash scenarios

Persist task state (DB/Redis) before enqueuing.

Use an acknowledgment callback to mark completion.

Run a compensating scheduled job that rescans “in‑progress” tasks older than a threshold and retries them.

Interview answer template

Do not use Executors shortcuts because of unbounded queues. Set parameters based on the formula, select CallerRunsPolicy for back‑pressure, integrate a dynamic thread‑pool framework such as Hippo4J or DynamicTp for real‑time adjustments, and add persistence plus compensation for reliability.

Advanced thinking – distributed deployment

If a single machine cannot handle the load, split the workload across multiple nodes, ensure task sharding, define state hand‑over on node failure, and apply global rate‑limiting to avoid overwhelming the SMS gateway.