Why Do 10,000 Virtual Threads Finish in 1 s While a Fixed Thread Pool Takes 50 s?

Conclusion

When most of the work is waiting (e.g., I/O), 10,000 tasks that each wait 1 s take about 50 s with a fixed pool of 200 threads because the pool processes the tasks in 50 batches (10000 ÷ 200 × 1 s = 50 s). Virtual threads can approximate a one‑task‑one‑thread model; waiting tasks yield their platform thread, allowing many waits to overlap, so total time is close to a single wait (1‑2 s).

Why the gap appears

Traditional Java threads map to OS threads, which are heavyweight; a fixed‑size pool limits concurrent tasks to the pool size. Virtual threads are managed by the JVM, are much lighter, and can be created in the thousands while only a small number of platform threads execute actual CPU work.

Code benchmark

The following program (requires JDK 21+) creates 10,000 tasks that each call Thread.sleep(1000) to simulate a 1‑second I/O operation. It runs the same workload with a fixed pool of 200 threads and with a virtual‑thread executor.

import java.time.Duration;
import java.time.Instant;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class VirtualThreadBenchmark {
    private static final int TASK_COUNT = 10_000;

    public static void main(String[] args) throws InterruptedException {
        run("Fixed Thread Pool", Executors.newFixedThreadPool(200));
        run("Virtual Threads", Executors.newVirtualThreadPerTaskExecutor());
    }

    private static void run(String name, ExecutorService executor) throws InterruptedException {
        Instant start = Instant.now();
        try (executor) {
            for (int i = 0; i < TASK_COUNT; i++) {
                executor.submit(() -> {
                    try {
                        Thread.sleep(1_000);
                    } catch (InterruptedException e) {
                        Thread.currentThread().interrupt();
                    }
                });
            }
            executor.shutdown();
            executor.awaitTermination(2, TimeUnit.MINUTES);
        }
        long elapsed = Duration.between(start, Instant.now()).toMillis();
        System.out.printf("%s: %.2f seconds%n", name, elapsed / 1_000.0);
    }
}

A typical run prints:

Fixed Thread Pool: 50.18 seconds
Virtual Threads: 1.12 seconds

The speedup comes from overlapping the 10,000 sleeps; CPU‑bound work would not see such a gain because the number of cores remains unchanged.

What virtual threads actually do

Platform threads are the real workers; virtual threads are lightweight tickets. When a virtual thread executes a blocking operation that supports virtual threads, it is detached from its platform thread, which can then run other tasks. Once the I/O completes, the virtual thread is rescheduled on an available platform thread.

Suitable use cases

Virtual threads excel in scenarios with many requests, heavy waiting, and little CPU work, such as:

Calling multiple remote services in parallel (e.g., fetching order, inventory, discount, logistics).

Database access where most time is spent waiting for SQL results (subject to connection‑pool limits).

Batch file processing (reading many small files or uploading/downloading objects).

Message consumption and gateway services where each task is simple but often blocked on network I/O.

Example: three remote calls with sleeps of 300 ms, 400 ms, and 500 ms run sequentially in ~1.2 s, but in parallel via virtual threads they finish in ~0.5 s, the duration of the longest call.

Limitations

Virtual threads cannot bypass downstream limits; a DB pool of 50 connections still caps concurrency.

CPU‑intensive tasks do not become faster; excessive concurrency may add scheduling overhead.

Do not treat virtual threads as a thread‑pool; they are cheap and usually created per task; use semaphores or other throttling when protecting shared resources.

Benchmarks that use only sleep illustrate the principle; production tests must monitor timeouts, memory, connection counts, and error rates.