9.5 Virtual Threads in Java 21 (Carrier Threads, Pinning, and Best Practices)

Virtual Threads in Java 21 (Carrier Threads, Pinning, and Best Practices)

Finalized as part of Project Loom in Java SE 21 (JEP 444), Virtual Threads represent the most significant architectural evolution of Java concurrency since Java 5. Virtual threads solve the thread scalability bottleneck by decoupling Java's thread abstraction from operating system kernel threads, enabling high-throughput concurrent applications written in the simple, synchronous, thread-per-request style.

---

1. The Thread Scalability Bottleneck: Platform vs. Virtual Threads

Historically, every java.lang.Thread was a Platform Thread (a thin wrapper around an OS kernel thread). This 1:1 architecture introduces severe limitations:

---

2. Carrier Threads and M:N Scheduling

Virtual threads are executed by mounting them onto underlying platform threads known as Carrier Threads:

• Carrier Pool: The JVM manages carrier threads using an internal FIFO ForkJoinPool.
• Default Parallelism: The number of carrier threads defaults to Runtime.getRuntime().availableProcessors() (configurable via -Djdk.virtualThreadScheduler.parallelism=N).
• Mounting & Continuation Mechanism:
  1. When a virtual thread executes code, it is mounted onto a carrier platform thread.
  2. When the virtual thread encounters a blocking operation (e.g., SocketChannel.read(), Thread.sleep(), BlockingQueue.take(), LockSupport.park()), the virtual thread yields its continuation.
  3. Its execution stack frames are copied from the carrier stack into the JVM heap, and the virtual thread unmounts.
  4. The carrier thread is immediately released to mount and execute another runnable virtual thread.
  5. When the blocking operation completes in the OS kernel, the JVM schedules the virtual thread continuation, mounting it on any available carrier thread (not necessarily the original one).

+-------------------------------------------------------------------------+
|                         MILLIONS OF VIRTUAL THREADS                     |
|  [VT 1: Active]    [VT 2: Blocked on I/O]    [VT 3: Active]    [VT 4...] |
+-------------------------------------------------------------------------+
          |                       | (Unmounted)           |
       (Mounted)                  v                    (Mounted)
+-------------------+   +-------------------+   +-------------------+
|  Carrier Thread 1 |   |  Carrier Thread 2 |   |  Carrier Thread 3 |  (ForkJoinPool)
|   (OS Thread 1)   |   |   (OS Thread 2)   |   |   (OS Thread 3)   |
+-------------------+   +-------------------+   +-------------------+

---

3. Creating and Launching Virtual Threads

Java 21 provides several APIs to instantiate virtual threads:

// Method 1: Instant launch helper
Thread vt1 = Thread.startVirtualThread(() -> {
    System.out.println("Running in VT: " + Thread.currentThread());
});

// Method 2: Fluent Builder API (Configured and started)
Thread vt2 = Thread.ofVirtual()
    .name("order-worker-", 0) // Names: order-worker-0, order-worker-1...
    .start(() -> System.out.println("Named VT running"));

// Method 3: Fluent Builder unstarted
Thread vt3 = Thread.ofVirtual()
    .name("batch-vt")
    .unstarted(() -> System.out.println("Unstarted VT"));
vt3.start();

// Method 4: ThreadFactory integration
ThreadFactory factory = Thread.ofVirtual().name("http-req-", 1).factory();
Thread vt4 = factory.newThread(() -> System.out.println("Factory created VT"));
vt4.start();

// Method 5: ExecutorService (One virtual thread per submitted task)
try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 10_000; i++) {
        int taskId = i;
        executor.submit(() -> {
            Thread.sleep(1000); // Unmounts cleanly; does not block OS thread
            return taskId;
        });
    }
} // Auto-closes: shuts down and awaits all 10,000 tasks

---

4. Invariants and Architectural Rules for the Exam

Virtual threads possess strict invariants that differ fundamentally from platform threads:

1. Permanent Daemon Status:
   • Virtual threads are always daemon threads. The JVM will never wait for virtual threads to finish when all non-daemon platform threads have terminated.
   • Calling vt.setDaemon(false) throws java.lang.IllegalArgumentException.
2. Fixed Normal Priority:
   • Virtual threads always have Thread.NORM_PRIORITY (5).
   • Calling vt.setPriority(int priority) is silently ignored; priority remains 5.
3. Fixed, Non-Configurable ThreadGroup:
   • Every virtual thread belongs to a single JVM-managed group named VirtualThreads, and the Thread.ofVirtual() builder offers no way to choose a different group. ThreadGroup is effectively inert in Java 21: destroy() is terminally deprecated and simply does nothing, and setMaxPriority() never changes the priority of virtual threads. Treat thread groups as a legacy diagnostic label, not a control surface.
4. Thread.currentThread().isVirtual():
   • Returns true if invoked on a virtual thread; false on platform threads.

Thread vt = Thread.ofVirtual().unstarted(() -> {});
System.out.println(vt.isDaemon());    // PRINTS: true
System.out.println(vt.isVirtual());   // PRINTS: true
System.out.println(vt.getPriority()); // PRINTS: 5

vt.setPriority(Thread.MAX_PRIORITY);  // Silently ignored; priority remains 5
// vt.setDaemon(false); // COMPILE OK, but THROWS IllegalArgumentException at runtime!

---

5. Thread Pinning: Causes, Diagnostics, and Remediation

What is Thread Pinning?

When a virtual thread blocks while pinned, it cannot unmount from its carrier thread. Consequently, the underlying OS carrier thread remains blocked and unavailable to execute other virtual threads, severely degrading throughput.

The Two Pinning Triggers:

1. Executing inside a synchronized block or method: Entering an intrinsic monitor lock binds the virtual thread stack frame to the OS carrier thread's C-runtime stack.
2. Executing a Native Method or Foreign Function: JNI native calls or Foreign Function & Memory (FFM) API calls.

Remediation: Replace synchronized with ReentrantLock

Unlike intrinsic monitors (synchronized), java.util.concurrent.locks.ReentrantLock uses LockSupport.park() for blocking. LockSupport.park() supports virtual thread continuations, allowing virtual threads to unmount cleanly without pinning!

// PINNING ANTI-PATTERN (Carrier thread is blocked!)
public class PinnedService {
    public synchronized String fetchData() { // synchronized causes pinning!
        try {
            return httpCall(); // Blocking I/O while holding monitor = PINNED!
        } catch (Exception e) { return ""; }
    }
}

// REFACTORED BEST PRACTICE (Cleanly unmounts carrier thread!)
public class SafeService {
    private final ReentrantLock lock = new ReentrantLock();

    public String fetchData() {
        lock.lock();
        try {
            return httpCall(); // Virtual thread unmounts cleanly during I/O!
        } catch (Exception e) {
            return "";
        } finally {
            lock.unlock();
        }
    }
}

Diagnosing Pinned Threads via JVM Flag

You can detect pinning at runtime by launching the JVM with: -Djdk.tracePinnedThreads=full (prints full stack trace) or -Djdk.tracePinnedThreads=short (prints one-line summary).

---

6. Best Practices and Anti-Patterns

Anti-Pattern 1: Pooling Virtual Threads

• Do NOT Pool Virtual Threads: Never use Executors.newFixedThreadPool(100) with virtual threads. Pooling virtual threads adds memory overhead and defeats their design.
• Best Practice: Create a new virtual thread per task using Executors.newVirtualThreadPerTaskExecutor() or Thread.ofVirtual().start().

Anti-Pattern 2: CPU-Bound Tasks on Virtual Threads

• Virtual threads provide zero throughput benefits for purely CPU-intensive computations (e.g., matrix math, video rendering, cryptographic hashing). Because CPU workloads cannot unmount, they monopolize carrier threads.
• Best Practice: Use platform thread pools or ForkJoinPool.commonPool() for CPU-bound tasks.

Anti-Pattern 3: Heavy ThreadLocal Caching

• Creating millions of virtual threads that each instantiate large cached objects in ThreadLocal can rapidly exhaust heap memory.
• Best Practice: Avoid large ThreadLocal caches in virtual threads; prefer ephemeral task parameters or modern features like Scoped Values (JEP 446).

---

7. Java 21 Concurrency Preview Overview

• Structured Concurrency (JEP 453 Preview): Coordinates groups of related subtasks running in concurrent virtual threads as a single transaction using StructuredTaskScope (ShutdownOnFailure for fail-fast or ShutdownOnSuccess for first-result racing).
• Scoped Values (JEP 446 Preview): Enables safe, immutable, and lightweight data sharing across threads without the memory overhead of ThreadLocal.