Concurrency Series Finale: Virtual Threads and Structured Concurrency

Why Platform Threads Are Expensive

In JDK 21 a platform thread is the traditional new Thread() that maps one‑to‑one to an OS kernel thread. Its cost comes from two hard resources:

Memory : each thread reserves a stack (default ~1 MB on 64‑bit HotSpot, configurable via -Xss). Creating 10 000 threads would consume ~10 GB of virtual memory, which is unrealistic on most machines.

Context switch : the OS kernel must save/restore registers, TLB and CPU caches on each switch, adding microsecond‑level latency and hidden cache‑miss costs. When the thread count far exceeds core count, CPU spends most time swapping contexts instead of doing work.

These costs impose a ceiling on the classic thread‑per‑request model (e.g., Tomcat takes a thread from a pool, performs DB/RPC calls, assembles a response, and returns). The model is easy to write and debug because the call stack is linear and try‑catch works normally.

In a typical 200 ms request where 190 ms is I/O wait, the thread spends 95 % of its time blocked while still holding a 1 MB stack and an OS thread.

To sustain 10 000 QPS with 200 ms latency you need about 2 000 concurrent threads (concurrency = QPS × response time). That already stresses memory and scheduling.

The actual CPU needed for those 2 000 blocked threads is only 2 000 × 5 % = 100 threads worth of work, leaving hardware idle but the thread count hitting the wall.

Two historical solutions:

Solution 1 – Thread pool + tuning : reuse threads to amortise creation cost using the formula cores × (1 + wait/compute). This spreads cost but does not remove the fundamental constraint that each blocked request occupies an OS thread.

Solution 2 – Async/Reactive : use CompletableFuture, Reactor, RxJava, Netty, etc., to register callbacks on I/O and release the thread. This achieves high concurrency with few threads but turns straight‑line code into fragmented callbacks, breaks stack traces, complicates debugging, invalidates ThreadLocal, and introduces “asynchronous contagion” where every caller must become async.

Virtual threads take a third path: they keep the simple synchronous programming model while making the thread itself cheap enough that blocking does not tie up an OS thread.

What Virtual Threads Are

A virtual thread is a java.lang.Thread instance scheduled by the JVM, not the OS. It still implements the same APIs, so existing code works unchanged, but it is no longer bound one‑to‑one with an OS thread.

Virtual Thread : the lightweight task‑level thread. Its stack lives on the Java heap as a continuation/stack chunk, starting at a few hundred bytes and growing on demand. Millions of virtual threads are feasible.

Carrier Thread : the underlying platform thread that actually runs bytecode. A virtual thread “mounts” onto a carrier thread to execute.

Scheduler : an internal ForkJoinPool (distinct from the common pool) that assigns virtual threads to carrier threads. Its parallelism defaults to the number of CPU cores and can be tuned via jdk.virtualThreadScheduler.parallelism.

The core mechanism consists of two actions:

Mount : the scheduler picks a carrier thread and moves the virtual thread’s stack frames onto the carrier’s OS stack to start execution.

Unmount : when the virtual thread reaches a blocking operation (e.g., socket.read(), Thread.sleep(), ReentrantLock, BlockingQueue.take()), the JVM saves the virtual thread’s stack back to the heap, releases the carrier thread, and later remounts the virtual thread onto any carrier when the I/O completes.

Thus a blocked virtual thread occupies only a few kilobytes on the heap, not a full OS thread. In the earlier 190 ms I/O example, the virtual thread is unmounted during the wait, freeing the carrier to serve thousands of other virtual threads.

All standard library blocking points (e.g., java.net.Socket, java.nio.channels, Thread.sleep(), java.util.concurrent locks/queues) have been retrofitted to recognize virtual threads and trigger unmounting, so existing synchronous code becomes transparent.

Version note : Virtual threads were previewed in JDK 19 (JEP 425), second preview in JDK 20 (JEP 436), and became a standard feature in JDK 21 (JEP 444). Production use requires JDK 21+; on 19/20 you must enable --enable-preview and the API may change.

Creating and Using Virtual Threads

The API is deliberately minimal, offering three typical usages:

Usage 1 : Thread.ofVirtual().start(() -> { … }) – create a single virtual thread and start it immediately.

// Start a virtual thread and join it
Thread vt = Thread.ofVirtual()
    .name("seckill-worker")
    .start(() -> {
        deductStock(productId); // block inside as needed, no async conversion
    });
vt.join(); // same as platform thread join

// Comparison with platform thread
Thread pt = Thread.ofPlatform().start(() -> deductStock(productId));

Usage 2 (recommended): Executors.newVirtualThreadPerTaskExecutor() – an ExecutorService that creates a fresh virtual thread for each submitted task (no pooling).

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (Long userId : userIds) { // assume 100 k user requests
        executor.submit(() -> {
            checkRiskControl(userId); // blocking external call
            deductStock(productId);   // blocking DB call
            createOrder(userId, productId); // blocking DB write
        });
    }
} // waits for all 100 k tasks to finish

Creating 100 k virtual threads is cheap (tens of MB on the heap) compared with a platform‑thread pool of the same size, which would likely OOM.

Usage 3 : Switch the whole web container to virtual threads. In Spring Boot 3.2+ (requires JDK 21) set spring.threads.virtual.enabled=true. Tomcat will then allocate one virtual thread per request, letting existing controller/service code run unchanged while gaining massive concurrency.

Pinning: When a Virtual Thread Gets Stuck to a Carrier

Normally a virtual thread unmounts on blocking, but two situations prevent unmounting, causing the virtual thread to be pinned to its carrier thread:

Blocking inside a synchronized block or method.

Calling a native method or an FFM (foreign function) where the native stack cannot be moved to the heap.

Pinning is dangerous only when the blocked region is long‑running, because the carrier thread then becomes blocked as well, defeating the scalability benefit.

Work‑around for JDK 21: replace synchronized with ReentrantLock, which the JVM can recognise and unmount correctly. In JDK 24 (JEP 491) the pinning problem for synchronized is solved, but native‑method pinning remains.

To diagnose pinning on JDK 21‑23, use -Djdk.tracePinnedThreads=full or the JFR event jdk.VirtualThreadPinned .

Usage Rules: No Pooling, Careful with ThreadLocal

Even though virtual threads are transparent, developers must adopt new mental models:

Rule 1 – Do not pool virtual threads. Pooling re‑introduces the concurrency ceiling. Each task should get its own virtual thread and be discarded after completion.

// WRONG: pool virtual threads, re‑introducing a limit
ExecutorService wrong = Executors.newFixedThreadPool(200, Thread.ofVirtual().factory());
// RIGHT: one virtual thread per task
ExecutorService right = Executors.newVirtualThreadPerTaskExecutor();

Virtual threads are cheap (just a small object), so pooling offers no benefit and actually harms scalability.

Rule 2 – Use ThreadLocal sparingly. Because millions of virtual threads may exist, storing large objects in a ThreadLocal can exhaust memory. Moreover, the traditional pattern of caching heavy objects (e.g., SimpleDateFormat) in a ThreadLocal loses its advantage because virtual threads are short‑lived.

Even though ThreadLocal works on virtual threads, you must still call remove() to avoid leaks, but the leak risk is lower because the thread ends and its locals are reclaimed.

A better alternative for context propagation is ScopedValue, discussed next.

Structured Concurrency

When you spawn many virtual threads, their lifetimes become scattered, leading to three classic bugs with plain Future usage:

Leak: if the first get() throws, the second task keeps running as an orphan.

Cannot cancel the whole group together.

Code structure does not express that the two tasks belong to the same request.

Structured concurrency solves this by tying the lifetime of child tasks to a lexical scope, similar to try‑with‑resources:

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Subtask<UserInfo> user = scope.fork(() -> queryUser(userId));
    Subtask<StockInfo> stock = scope.fork(() -> queryStock(productId));
    scope.join().throwIfFailed(); // any failure cancels the other
    return placeOrder(user.get(), stock.get());
} // exiting the block guarantees both subtasks are finished

Other strategies like ShutdownOnSuccess cancel remaining tasks as soon as one succeeds, useful for race‑style queries.

try (var scope = new StructuredTaskScope.ShutdownOnSuccess<StockInfo>()) {
    scope.fork(() -> queryFromLocalCache(productId));
    scope.fork(() -> queryFromRedis(productId));
    scope.join();
    return scope.result(); // first successful result
}

Version note : Structured concurrency is still preview in JDK 21‑26 and its API has changed across versions. Always check the target JDK documentation and enable --enable-preview when compiling.

ScopedValue: Context Propagation for Virtual Threads

Because ThreadLocal suffers from mutable state and manual cleanup, JDK introduced ScopedValue to bind a value to a lexical scope:

private static final ScopedValue<UserInfo> CURRENT_USER = ScopedValue.newInstance();

ScopedValue.where(CURRENT_USER, user).run(() -> {
    placeOrder(productId); // any code inside can read CURRENT_USER
});

public void deductStock(Long productId) {
    UserInfo user = CURRENT_USER.get(); // read‑only
}

Key differences from ThreadLocal:

Immutable – cannot be changed after binding.

Lifetime defined by the surrounding code block; automatically cleared when the block exits.

Designed for massive virtual‑thread scenarios: the value is shared, not copied per thread.

On JDK 21 production code, ThreadLocal remains usable (it works on virtual threads) as long as you avoid storing heavy objects.

Virtual Thread vs Platform Thread‑Pool vs Reactive: Model Comparison

Programming model : Platform thread‑pool – synchronous blocking, easy to read; Virtual thread – synchronous blocking, easy to read ; Reactive – async callbacks/streams, high mental load.

Cost per "thread" : Platform – ~1 MB stack + OS thread; Virtual – heap object a few hundred bytes, grows on demand; Reactive – no thread concept, tasks are callbacks.

Supported concurrency : Platform – hundreds to thousands; Virtual – tens of thousands to millions ; Reactive – very high.

Scheduler : Platform – OS kernel; Virtual – JVM (dedicated ForkJoinPool); Reactive – event loop.

Stack trace / debugging : Platform – complete and clear; Virtual – complete and clear ; Reactive – fragmented, hard to debug. ThreadLocal support: Platform – normal; Virtual – usable but avoid heavy objects; Reactive – essentially ineffective.

Best fit : Platform – CPU‑bound tasks; Virtual – IO‑bound business services ; Reactive – extreme throughput, streaming, back‑pressure scenarios.

Migration cost : Platform – —; Virtual – very low (configuration only) ; Reactive – high (full code‑base rewrite).

Selection guidance :

IO‑bound typical web/microservice backends → use virtual threads on JDK 21+ for “reactive‑level throughput with synchronous readability”.

CPU‑bound heavy computation → stay with a platform thread pool sized to core count or use Fork/Join.

Existing high‑performing reactive systems → no need to rewrite; however, new projects that only need concurrency should consider virtual threads first.

Virtual threads also let you revert CompletableFuture chains back to plain sequential code, improving readability when the only purpose of the chain was to avoid blocking.

Full Series Recap

The 14‑article series can be viewed as four layers of questions:

Foundations : What causes concurrency problems? (atomicity, visibility, ordering – JMM, happens‑before).

Basic tools : How to guarantee atomicity and visibility? (volatile, synchronized, CAS, AQS, locks, concurrent collections).

Engineering practice : How to write correct concurrent code? (deadlock conditions, ThreadLocal pitfalls, thread‑pool tuning, concurrent containers).

Advanced async : How to make threads work harder? (CompletableFuture, Fork/Join, and finally virtual threads which remove the “threads are expensive” premise).

Virtual threads do not eliminate the core challenges of shared mutable state – atomicity, visibility, and deadlocks still exist – but they remove the cost barrier that previously forced developers into complex async patterns.

Key takeaways:

All concurrency issues stem from “shared mutable state”; choose between no sharing , immutability , or coordination (locks, CAS, etc.).

Identify a concurrency primitive by the problem it solves (volatile → visibility, CAS → atomicity, locks → all three) and its trade‑offs.

Understanding the underlying mechanisms (CPU caches, memory barriers, JMM) turns concurrency from mysticism into reasoned engineering.