diff --git a/docs/src/main/asciidoc/resteasy-reactive-virtual-threads.adoc b/docs/src/main/asciidoc/resteasy-reactive-virtual-threads.adoc new file mode 100644 index 0000000000000..7f22696688f97 --- /dev/null +++ b/docs/src/main/asciidoc/resteasy-reactive-virtual-threads.adoc @@ -0,0 +1,316 @@ +//// +This guide is maintained in the main Quarkus repository +and pull requests should be submitted there: +https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc +//// +[[resteasy-reactive-virtual-threads]] += Use virtual threads in REST applications +include::_attributes.adoc[] +:diataxis-type: howto +:categories: web, core +:summary: How to use virtual threads in a REST application + +In this guide, we see how you can use virtual threads in a REST application. +Because virtual threads are all about I/O, we will also use the REST client. + +== Prerequisites + +include::{includes}/prerequisites.adoc[] + +== Architecture + +The application built into this guide is quite simple. +It calls a weather service for two cities (Valence, France, and Athens, Greece) and decides the best place based on the current temperature. + + +== Create the Maven project + +First, we need a new project. Create a new project with the following command: + +:create-app-artifact-id: rest-virtual-threads +:create-app-extensions: resteasy-reactive-jackson,quarkus-rest-client-reactive-jackson +include::{includes}/devtools/create-app.adoc[] + +This command generates a new project importing the RESTEasy Reactive, Reactive REST client, and https://github.com/FasterXML/jackson[Jackson] extensions, +and in particular, adds the following dependencies: + +[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"] +.pom.xml +---- + + io.quarkus + quarkus-resteasy-reactive-jackson + + + io.quarkus + quarkus-rest-client-reactive-jackson + +---- + +[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"] +.build.gradle +---- +implementation("io.quarkus:quarkus-resteasy-reactive-jackson") +implementation("quarkus-rest-client-reactive-jackson") +---- + +[NOTE] +==== +You might wonder why we use _reactive_ extensions. +Virtual threads have limitations, and we can only integrate them properly when using the reactive extensions. +No worries: your code will be written 100% in a synchronous / imperative style. + +Check the xref:./virtual-threads.adoc#why-not[virtual thread reference guide] for details. +==== + +== Prepare the `pom.xml` file + +We need to customize the `pom.xml` file to use virtual threads. + +1) Locate the line with `17`, and replace it with: + +[source, xml] +---- + 20 +---- + +2) In the maven-compiler-plugin configuration, add the `--enable-preview` arg (only if you use Java 19 or Java 20) + +[source, xml] +---- + + maven-compiler-plugin + ${compiler-plugin.version} + + + -parameters + --enable-preview + + + +---- + +3) In the maven-surefire-plugin and maven-failsafe-plugin configurations, add the following `argLine` parameter: + +[source, xml] +---- + + maven-surefire-plugin + ${surefire-plugin.version} + + + org.jboss.logmanager.LogManager + ${maven.home} + + --enable-preview -Djdk.tracePinnedThreads + + + + maven-failsafe-plugin + ${surefire-plugin.version} + + + + integration-test + verify + + + + ${project.build.directory}/${project.build.finalName}-runner + org.jboss.logmanager.LogManager + ${maven.home} + + --enable-preview -Djdk.tracePinnedThreads + + + + +---- + + +The `-Djdk.tracePinnedThreads` will detect pinned carrier threads while running tests (See xref:./virtual-threads.adoc#pinning[the virtual thread reference guide for details]). + +[IMPORTANT] +.--enable-preview and Java 21 +==== +The `--enable-preview` flag is only required if you use Java 19 or 20. +==== + +== Create the weather client + +This section is not about virtual threads. +Because we need to do some I/O to demonstrate virtual threads usage, we need a client doing I/O operations. +In addition, the reactive REST client is virtual thread friendly: it does not pin and handle propagation correctly. + +Create the `src/main/java/org/acme/WeatherService.java` class with the following content: + +[source, java] +---- +package org.acme; + +import com.fasterxml.jackson.annotation.JsonProperty; +import io.quarkus.rest.client.reactive.ClientQueryParam; +import jakarta.ws.rs.GET; +import org.eclipse.microprofile.rest.client.inject.RegisterRestClient; +import org.jboss.resteasy.reactive.RestQuery; + +@RegisterRestClient(baseUri = "https://api.open-meteo.com/v1/forecast") +public interface WeatherService { + + @GET + @ClientQueryParam(name = "current_weather", value = "true") + WeatherResponse getWeather(@RestQuery double latitude, @RestQuery double longitude); + + + record WeatherResponse(@JsonProperty("current_weather") Weather weather) { + // represents the response + } + + record Weather(double temperature, double windspeed) { + // represents the inner object + } +} +---- + +This class models the HTTP interaction with the weather service. +Read more about the rest client in the dedicated xref:./rest-client-reactive.adoc[guide]. + +== Create the HTTP endpoint + +Now, create the `src/main/java/org/acme/TheBestPlaceToBeResource.java` class with the following content: + +[source, java] +---- +package org.acme; + +import io.smallrye.common.annotation.RunOnVirtualThread; +import jakarta.ws.rs.GET; +import jakarta.ws.rs.Path; +import org.eclipse.microprofile.rest.client.inject.RestClient; + +@Path("/") +public class TheBestPlaceToBeResource { + + static final double VALENCE_LATITUDE = 44.9; + static final double VALENCE_LONGITUDE = 4.9; + + static final double ATHENS_LATITUDE = 37.9; + static final double ATHENS_LONGITUDE = 23.7; + + @RestClient WeatherService service; + + @GET + @RunOnVirtualThread <1> + public String getTheBestPlaceToBe() { + var valence = service.getWeather(VALENCE_LATITUDE, VALENCE_LONGITUDE).weather().temperature(); + var athens = service.getWeather(ATHENS_LATITUDE, ATHENS_LONGITUDE).weather().temperature(); + + // Advanced decision tree + if (valence > athens && valence <= 35) { + return "Valence! (" + Thread.currentThread() + ")"; + } else if (athens > 35) { + return "Valence! (" + Thread.currentThread() + ")"; + } else { + return "Athens (" + Thread.currentThread() + ")"; + } + } +} +---- +<1> Instructs Quarkus to invoke this method on a virtual thread + +== Run the application in dev mode + +Make sure that you use OpenJDK and JVM versions supporting virtual thread and launch the _dev mode_ with `./mvnw quarkus:dev`: + +[source, shell] +---- +> java --version +openjdk 20.0.1 2023-04-18 <1> +OpenJDK Runtime Environment Temurin-20.0.1+9 (build 20.0.1+9) +OpenJDK 64-Bit Server VM Temurin-20.0.1+9 (build 20.0.1+9, mixed mode) + +> ./mvnw quarkus:dev <2> +---- +<1> Must be 19+ +<2> Launch the dev mode + +Then, in a browser, open http://localhost:8080. +You should get something like: + +[source, text] +---- +Valence! (VirtualThread[#144]/runnable@ForkJoinPool-1-worker-6) +---- + +As you can see, the endpoint runs on a virtual thread. + +It's essential to understand what happened behind the scene: + +1. Quarkus creates the virtual thread to invoke your endpoint (because of the `@RunOnVirtualThread` annotation). +2. When the code invokes the rest client, the virtual thread is blocked, BUT the carrier thread is not blocked (that's the virtual thread _magic touch_). +3. Once the first invocation of the rest client completes, the virtual thread is rescheduled and continues its execution. +4. The second rest client invocation happens, and the virtual thread is blocked again (but not the carrier thread). +5. Finally, when the second invocation of the rest client completes, the virtual thread is rescheduled and continues its execution. +6. The method returns the result. The virtual thread terminates. +7. The result is captured by Quarkus and written in the HTTP response + +== Verify pinning using tests + +In the `pom.xml,` we added an `argLine` argument to the surefire and failsafe plugins: + +[source, xml] +---- +--enable-preview -Djdk.tracePinnedThreads +---- + +The `-Djdk.tracePinnedThreads` dumps the stack trace if a virtual thread cannot be _unmounted_ smoothly (meaning that it blocks the carrier thread). +That's what we call _pinning_ (more info in xref:./virtual-threads.adoc#pinning[the virtual thread reference guide]). + +We recommend enabling this flag in tests. +Thus, you can check that your application behaves correctly when using virtual threads. +Just check your log after having run the test. +If you see a stack trace... better check what's wrong. +If your code (or one of your dependencies) pins, it might be better to use regular worker thread instead. + +Create the `src/test/java/org/acme/TheBestPlaceToBeResourceTest.java` class with the following content: + +[source, java] +---- +package org.acme; + +import io.quarkus.test.junit.QuarkusTest; +import io.restassured.RestAssured; +import org.junit.jupiter.api.Test; + +@QuarkusTest +class TheBestPlaceToBeResourceTest { + + @Test + void verify() { + RestAssured.get("/") + .then() + .statusCode(200); + } + +} +---- + +It is a straightforward test, but at least it will detect if our application is pinning. +Run the test with either: + +- `r` in dev mode (using continuous testing) +- `./mvnw test` + +As you will see, it does not pin - no stack trace. +It is because the reactive REST client is implemented in a virtual-thread-friendly way. + +The same approach can be used with integration tests. + +== Conclusion + +This guide shows how you can use virtual threads with RESTEasy Reactive and the reactive REST client. +Learn more about virtual threads support on: + +- xref:./messaging-virtual-threads.adoc[@RunOnVirtualThread in messaging applications] (this guide covers Apache Kafka) +- xref:./grpc-virtual-threads.adoc[@RunOnVirtualThread in gRPC services] +- xref:./virtual-threads.adoc[the virtual thread reference guide] (include native compilation and containerization) \ No newline at end of file diff --git a/docs/src/main/asciidoc/resteasy-reactive.adoc b/docs/src/main/asciidoc/resteasy-reactive.adoc index 2afd51743055b..80161aa94a33c 100644 --- a/docs/src/main/asciidoc/resteasy-reactive.adoc +++ b/docs/src/main/asciidoc/resteasy-reactive.adoc @@ -1846,6 +1846,7 @@ how to use it. Here are some more advanced topics that you may not need to know about initially, but could prove useful for more complex use cases. +[#execution-model-blocking-non-blocking] === Execution model, blocking, non-blocking [[execution-model]] @@ -1853,7 +1854,7 @@ could prove useful for more complex use cases. RESTEasy Reactive is implemented using two main thread types: - Event-loop threads: which are responsible, among other things, for reading bytes from the HTTP request and - writing bytes back to the HTTP response +writing bytes back to the HTTP response - Worker threads: they are pooled and can be used to offload long-running operations The event-loop threads (also called IO threads) are responsible for actually performing all the IO diff --git a/docs/src/main/asciidoc/virtual-threads.adoc b/docs/src/main/asciidoc/virtual-threads.adoc index 7b43585d445ae..851c1f8ce682d 100644 --- a/docs/src/main/asciidoc/virtual-threads.adoc +++ b/docs/src/main/asciidoc/virtual-threads.adoc @@ -3,102 +3,172 @@ This guide is maintained in the main Quarkus repository and pull requests should be submitted there: https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc //// -= Writing simpler reactive REST services with Quarkus Virtual Thread support - +[[virtual-threads]] += Virtual Thread support reference include::_attributes.adoc[] +:diataxis-type: reference +:categories: core :resteasy-reactive-api: https://javadoc.io/doc/io.quarkus.resteasy.reactive/resteasy-reactive/{quarkus-version} -:resteasy-reactive-common-api: https://javadoc.io/doc/io.quarkus.resteasy.reactive/resteasy-reactive-common/{quarkus-version} :runonvthread: https://javadoc.io/doc/io.smallrye.common/smallrye-common-annotation/latest/io/smallrye/common/annotation/RunOnVirtualThread.html :blockingannotation: https://javadoc.io/doc/io.smallrye.common/smallrye-common-annotation/latest/io/smallrye/common/annotation/Blocking.html :vthreadjep: https://openjdk.org/jeps/425 :thread: https://docs.oracle.com/en/java/javase/18/docs/api/java.base/java/lang/Thread.html -:mutiny-vertx-sql: https://smallrye.io/smallrye-mutiny-vertx-bindings/2.26.0/apidocs/io/vertx/mutiny/sqlclient/package-summary.html :pgsql-driver: https://javadoc.io/doc/org.postgresql/postgresql/latest/index.html -This guide explains how to benefit from Java 19 virtual threads when writing REST services in Quarkus. - -[TIP] -==== -This is the reference guide for using virtual threads to write reactive REST services. -Please refer to the xref:rest-json.adoc[Writing JSON REST services guides] for a lightweight introduction to reactive REST -services and to the xref:resteasy-reactive.adoc[Writing REST Services with RESTEasy Reactive] guide for a detailed presentation. -==== +This guide explains how to benefit from Java 19+ virtual threads in Quarkus application. -== What are virtual threads ? +== What are virtual threads? === Terminology OS thread:: -A "thread-like" data-structure managed by the Operating System. +A "thread-like" data structure managed by the Operating System. Platform thread:: -Up until Java 19, every instance of the link:{thread}[Thread] class was a platform thread, that is, a wrapper around an OS thread. -Creating a platform threads creates an OS thread, blocking a platform thread blocks an OS thread. +Until Java 19, every instance of the link:{thread}[Thread] class was a platform thread, a wrapper around an OS thread. +Creating a platform thread creates an OS thread, and blocking a platform thread blocks an OS thread. Virtual thread:: Lightweight, JVM-managed threads. They extend the link:{thread}[Thread] class but are not tied to one specific OS thread. Thus, scheduling virtual threads is the responsibility of the JVM. Carrier thread:: -A platform thread used to execute a virtual thread is called a carrier. -This isn't a class distinct from link:{Thread}[Thread] or VirtualThread but rather a functional denomination. +A platform thread used to execute a virtual thread is called a **carrier** thread. +It isn't a class distinct from link:{Thread}[Thread] or `VirtualThread` but rather a functional denomination. === Differences between virtual threads and platform threads -We will give a brief overview of the topic here, please refer to the link:{vthreadjep}[JEP 425] for more information. +We will give a brief overview of the topic here; please refer to the link:{vthreadjep}[JEP 425] for more information. -Virtual threads are a feature available since Java 19 aiming at providing a cheap alternative to platform threads for I/O-bound workloads. +Virtual threads are a feature available since Java 19, aiming at providing a cheap alternative to platform threads for I/O-bound workloads. Until now, platform threads were the concurrency unit of the JVM. They are a wrapper over OS structures. -This means that creating a Java platform thread actually results in creating a "thread-like" structure in your operating system. +Creating a Java platform thread creates a "thread-like" structure in your operating system. -Virtual threads on the other hand are managed by the JVM. In order to be executed, they need to be mounted on a platform thread -(which acts as a carrier to that virtual thread). +Virtual threads, on the other hand, are managed by the JVM. To be executed, they need to be mounted on a platform thread (which acts as a carrier to that virtual thread). As such, they have been designed to offer the following characteristics: Lightweight :: Virtual threads occupy less space than platform threads in memory. -Hence, it becomes possible to use more virtual threads than platform threads simultaneously without blowing up the heap. -By default, platform threads are created with a stack of about 1 MB where virtual threads stack is "pay-as-you-go". -You can find these numbers along with other motivations for virtual threads in this presentation given by the lead developer of project Loom: https://youtu.be/lIq-x_iI-kc?t=543. +Hence, it becomes possible to use more virtual threads than platform threads simultaneously without blowing up the memory. +By default, platform threads are created with a stack of about 1 MB, whereas virtual threads stack is "pay-as-you-go." +You can find these numbers and other motivations for virtual threads in this https://youtu.be/lIq-x_iI-kc?t=543[presentation] given by the lead developer of project Loom (the project that added the virtual thread support to the JVM). Cheap to create:: Creating a platform thread in Java takes time. -Currently, techniques such as pooling where threads are created once then reused are strongly encouraged to minimize the -time lost in starting them (as well as limiting the maximum number of threads to keep memory consumption low). +Currently, techniques such as pooling, where threads are created once and then reused, are strongly encouraged to minimize the time lost in starting them (as well as limiting the maximum number of threads to keep memory consumption low). Virtual threads are supposed to be disposable entities that we create when we need them, -it is discouraged to pool them or to reuse them for different tasks. +it is discouraged to pool them or reuse them for different tasks. -Cheap to block:: When performing blocking I/O, the underlying OS thread wrapped by the Java platform thread is put in a -wait queue and a context switch occurs to load a new thread context onto the CPU core. This operation takes time. -Since virtual threads are managed by the JVM, no underlying OS thread is blocked when they perform a blocking operation. -Their state is simply stored in the heap and another Virtual thread is executed on the same Java platform thread. +Cheap to block:: When performing blocking I/O, the underlying OS thread wrapped by the Java platform thread is put in a wait queue, and a context switch occurs to load a new thread context onto the CPU core. This operation takes time. +Since the JVM manages virtual threads, no underlying OS thread is blocked when they perform a blocking operation. +Their state is stored in the heap, and another virtual thread is executed on the same Java platform (carrier) thread. +=== The Continuation Dance +As mentioned above, the JVM schedules the virtual threads. These virtual threads are mounted on carrier threads. The scheduling comes with a pinch of magic. When the virtual thread attempts to use blocking I/O, the JVM _transforms_ this call into a non-blocking one, unmounts the virtual thread, and mounts another virtual thread on the carrier thread. When the I/O completes, the _waiting_ virtual thread becomes eligible again and will be re-mounted on a carrier thread to continue its execution. For the user, all this dance is invisible. Your synchronous code is executed asynchronously. + +Note that the virtual thread may not be re-mounted on the same carrier thread. + +[[cpu-bound]] === Virtual threads are useful for I/O-bound workloads only -We now know that we can create way more virtual threads than platform threads. One could be tempted to use virtual threads -to perform long computations (CPU-bound workload). -This is useless if not counterproductive. -CPU-bound doesn't consist in quickly swapping threads while they need to wait for the completion of an I/O but in leaving -them attached to a CPU-core to actually compute something. -In this scenario, it is useless to have thousands of threads if we have tens of CPU-cores, virtual threads won't enhance -the performance of CPU-bound workloads. +We now know we can create more virtual threads than platform threads. One could be tempted to use virtual threads to perform long computations (CPU-bound workload). +It is useless and counterproductive. +CPU-bound doesn't consist of quickly swapping threads while they need to wait for the completion of an I/O, but in leaving +them attached to a CPU core to compute something. +In this scenario, it is worse than useless to have thousands of threads if we have tens of CPU cores, virtual threads won't enhance the performance of CPU-bound workloads. +Even worse, when running a CPU-bound workload on a virtual thread, the virtual thread monopolizes the carrier thread on which it is mounted. +It will either reduce the chance for the other virtual thread to run or will start creating new carrier threads, leading to high memory usage. + +== Run code on virtual threads using @RunOnVirtualThread + +In Quarkus, the support of virtual thread is implemented using the link:{runonvthread}[@RunOnVirtualThread] annotation. +This section briefly overviews the rationale and how to use it. +There are dedicated guides for extensions supporting that annotation, such as // TODO. + +[[why-not]] +=== Why not run everything on virtual threads? +As mentioned above, not everything can run safely on virtual threads. +The risk of **monopolization** can lead to high-memory usage. +Also, there are situations where the virtual thread cannot be unmounted from the carrier thread. +This is called **pinning**. +Finally, some libraries use `ThreadLocal` to store and reuse objects. +Using virtual threads with these libraries will lead to massive allocation, as the intentionally pooled objects will be instantiated for every (disposable and generally short-lived) virtual thread. + +As of today, it is not possible to use virtual threads in a carefree manner. +Following such a laissez-faire approach could quickly lead to memory and resource starvation issues. +Thus, Quarkus uses an explicit model until the aforementioned issues disappear (as the Java ecosystem matures). +It is also the reason why _reactive_ extensions have the virtual thread support, and rarely the _classic_ ones. +We need to know when to dispatch on a virtual thread. + +It is essential to understand that these issues are not Quarkus limitations or bugs but are due to the current state of the Java ecosystem which needs to evolve to become virtual thread friendly. + +NOTE: To learn more about the internal design and choices, check the https://dl.acm.org/doi/10.1145/3583678.3596895[Considerations for integrating virtual threads in a Java framework: a Quarkus example in a resource-constrained environment] paper. + +[[monopolization]] +==== Monopolization cases +The monopolization has been explained in the xref:cpu-bound[Virtual threads are useful for I/O-bound workloads only] section. +When running long computations, we do not allow the JVM to unmount and switch to another virtual thread until the virtual thread terminates. +Indeed, the current scheduler does not support preempting tasks. + +This monopolization can lead to the creation of new carrier threads to execute other virtual threads. +Creating carrier threads results in creating platform threads. +So, there is a memory cost associated with this creation. + +Suppose you run in a constrained environment, such as containers. In that case, monopolization can quickly become a concern, as the high memory usage can lead to out-of-memory issues and container termination. +The memory usage may be higher than with regular worker threads because of the inherent cost of the scheduling and virtual threads. + +[[pinning]] +==== Pinning cases +The promise of "cheap blocking" might not always hold: a virtual thread might _pin_ its carrier on certain occasions. +The platform thread is blocked in this situation, precisely as it would have been in a typical blocking scenario. + +According to link:{vthreadjep}[JEP 425] this can happen in two situations: + +- when a virtual thread performs a blocking operation inside a `synchronized` block or method +- when it executes a blocking operation inside a native method or a foreign function + +It can be reasonably easy to avoid these situations in your code, but verifying every dependency you use is hard. +Typically, while experimenting with virtual threads, we realized that old versions of the link:{pgsql-driver}[postgresql-JDBC driver] +results in frequent pinning. +Most JDBC drivers still pin the carrier thread. +Even worse, lots of widespread libraries are pinning and would require code changes. +[[pooling]] +==== The pooling case +Some libraries are using `ThreadLocal` as an object pooling mechanism. +Extremely popular libraries like https://github.com/FasterXML/jackson-core/issues/919[Jackson] and Netty assume that the application uses a limited number of threads, which are recycled (using a thread pool) to run multiple (unrelated but sequential) tasks. -== Bringing virtual threads to reactive REST services -Since virtual threads are disposable entities, the fundamental idea of quarkus-loom is to offload the execution of an -endpoint handler on a new virtual thread instead of running it on an event-loop (in the case of RESTeasy-reactive) or a -platform worker thread. +This pattern has multiple advantages, such as: + +- Allocation benefit: heavy objects are only allocated once per thread, but because the number of these threads was intended to be limited, it would not use too much memory. +- Thread safety: only one thread can access the object stored in the thread local - preventing concurrent accesses. + +However, this pattern is counter-productive when using virtual threads. +Virtual threads are not pooled and generally short-lived. +So, instead of a few of them, we now have many of them. +For each of them, the object stored in the `ThreadLocal` is created (often large and expensive) and won't be reused, as the virtual thread is not pooled (and won't be used to run another task once the execution completes). +This problem leads to high memory usage. +Unfortunately, it requires sophisticated code changes in the libraries themselves. + +=== Use @RunVirtualThread with RESTEasy Reactive + +This section shows a brief example of using the link:{runonvthread}[@RunOnVirtualThread] annotation. +It also explains the various development and execution models offered by Quarkus. + +The `@RunOnVirtualThread` annotation instructs Quarkus to invoke the annotated method on a **new** virtual thread instead of the current one. +Quarkus handles the creation of the virtual thread and the offloading. + +Since virtual threads are disposable entities, the fundamental idea of `@RunOnVirtualThread` is to offload the execution of an endpoint handler on a new virtual thread instead of running it on an event-loop or worker thread (in the case of RESTEasy Reactive). To do so, it suffices to add the link:{runonvthread}[@RunOnVirtualThread] annotation to the endpoint. -If the JDK is compatible (Java 19 or later versions) then the endpoint will be offloaded to a virtual thread. -It will then be possible to perform blocking operations without blocking the platform thread upon which the virtual -thread is mounted. +If the Java Virtual Machine used to **run** the application provides virtual thread support (so, Java 19 or later versions), then the endpoint execution is offloaded to a virtual thread. +It will then be possible to perform blocking operations without blocking the platform thread upon which the virtual thread is mounted. -This annotation can only be used in conjunction with endpoints annotated with link:{blockingannotation}[@Blocking] or +In the case of RESTEasy Reactive, this annotation can only be used on endpoints annotated with link:{blockingannotation}[@Blocking] or considered blocking because of their signature. You can visit xref:resteasy-reactive.adoc#execution-model-blocking-non-blocking[Execution model, blocking, non-blocking] for more information. -=== Getting started +==== Get started with virtual threads with RESTEasy Reactive -Add the following import to your build file: +Add the following dependency to your build file: [source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"] .pom.xml @@ -115,7 +185,7 @@ Add the following import to your build file: implementation("io.quarkus:quarkus-resteasy-reactive") ---- -You also need to make sure that you are using the version 19 of Java, this can be enforced in your pom.xml file with the following: +Then, you also need to make sure that you are using the version 19+ of Java, this can be enforced in your pom.xml file with the following: [source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"] .pom.xml @@ -126,29 +196,33 @@ You also need to make sure that you are using the version 19 of Java, this can b ---- -Virtual threads are still an experimental feature, you need to start your application with the `--enable-preview` flag: +Finally, until Java 21, you need to configure your compiler plugin with the `--enable-preview` flag. +If you use Maven, make sure that the configuration of the Maven compiler plugin is the following: -[source, bash] +[source, xml] ---- -java --enable-preview -jar target/quarkus-app/quarkus-run.jar + + maven-compiler-plugin + ${compiler-plugin.version} + + + --enable-preview + -parameters + + + ---- -The example below shows the differences between three endpoints, all of them querying a fortune in the database then +==== Three development and execution models + +The example below shows the differences between three endpoints, all of them querying a _fortune_ in the database then returning it to the client. - the first one uses the traditional blocking style, it is considered blocking due to its signature. -- the second one uses Mutiny reactive streams in a declarative style, it is considered non-blocking due to its signature. -- the third one uses Mutiny reactive streams in a synchronous way, since it doesn't return a "reactive type" it is +- the second one uses Mutiny, it is considered non-blocking due to its signature. +- the third one uses Mutiny but in a synchronous way, since it doesn't return a "reactive type" it is considered blocking and the link:{runonvthread}[@RunOnVirtualThread] annotation can be used. -When using Mutiny, alternative "xAndAwait" methods are provided to be used with virtual threads. -They ensure that waiting for the completion of the I/O will not "pin" the carrier thread and deteriorate performance. -Pinning is a phenomenon that we describe in xref:Pinning cases[this section]. - - -In other words, the mutiny environment is a safe environment for virtual threads. -The guarantees offered by Mutiny are detailed later. - [source,java] ---- package org.acme.rest; @@ -167,9 +241,12 @@ import java.util.Random; @Path("") public class FortuneResource { + @Inject FortuneRepository repository; + @GET @Path("/blocking") public Fortune blocking() { + // Runs on a worker (platform) thread var list = repository.findAllBlocking(); return pickOne(list); } @@ -177,6 +254,7 @@ public class FortuneResource { @GET @Path("/reactive") public Uni reactive() { + // Runs on the event loop return repository.findAllAsync() .map(this::pickOne); } @@ -185,6 +263,7 @@ public class FortuneResource { @Path("/virtual") @RunOnVirtualThread public Fortune virtualThread() { + // Runs on a virtual thread var list = repository.findAllAsyncAndAwait(); return pickOne(list); } @@ -192,306 +271,185 @@ public class FortuneResource { } ---- -=== Simplifying complex logic -The previous example is trivial and doesn't capture how imperative style can simplify complex reactive operations. -Below is a more complex example. -The endpoints must now fetch all the fortunes in the database, then append a quote to each fortune before finally returning -the result to the client. +The following table summarizes the options: +|=== +|Model |Example of signature |Pros |Cons +|Synchronous code on worker thread +|`Fortune blocking()` +|Simple code +|Use worker thread (limit concurrency) -[source,java] ----- -package org.acme.rest; +|Reactive code on event loop +|`Uni reactive()` +|High concurrency and low resource usage +|More complex code -import org.acme.fortune.model.Fortune; -import org.acme.fortune.repository.FortuneRepository; -import io.smallrye.common.annotation.RunOnVirtualThread; -import io.smallrye.mutiny.Uni; +|Synchronous code on virtual thread +|`@RunOnVirtualThread Fortune vt()` +|Simple code +|Risk of pinning, monopolization and under-efficient object pooling +|=== -import jakarta.ws.rs.GET; -import jakarta.ws.rs.Path; -import java.util.List; -import java.util.Random; +Note that all three models can be used in a single application. +== Use virtual thread friendly clients -@Path("") -public class FortuneResource { +As mentioned in the href:why-not[Why not run everything on virtual threads?] section, the Java ecosystem is not entirely ready for virtual threads. +So, you need to be careful, especially when using a libraries doing I/O. - private final FortuneRepository repository; +Fortunately, Quarkus provides a massive ecosystem that is ready to be used in virtual threads. +Mutiny, the reactive programming library used in Quarkus, and the Vert.x Mutiny bindings provides the ability to write blocking code (so, no fear, no learning curve) which do not pin the carrier thread. - public Uni> getQuotesAsync(int size){ - //... - //asynchronously returns a list of quotes from an arbitrary source - } +As a result: - @GET - @Path("/quoted-blocking") - public List getAllQuotedBlocking() { - // we get the list of fortunes - var fortunes = repository.findAllBlocking(); - - // we get the list of quotes - var quotes = getQuotesAsync(fortunes.size()).await().indefinitely(); - - // we append each quote to each fortune - for(int i=0; i < fortunes.size(); i ++){ - fortunes.get(i).title += " - " + quotes.get(i); - } - return fortunes; - } +1. Quarkus extensions providing blocking APIs on top of reactive APIs can be used in virtual threads. +This includes the reactive rest client, the redis client, the mailer... +2. API returning `Uni` can be used directly using `uni.await().atMost(...)`. It blocks the virtual thread, without blocking the carrier thread, and also improves the resilience of your application with an easy (non-blocking) timeout support. +3. If you use a https://smallrye.io/smallrye-mutiny-vertx-bindings/latest/[Vert.x client using the Mutiny bindings], use the `andAwait()` methods which block until you get the result without pinning the carrier thread. It includes all the reactive SQL drivers. - @GET - @Path("/quoted-reactive") - public Uni> getAllQuotedReactive() { - // we first fetch the list of resource and we memoize it - // to avoid fetching it again everytime need it - var fortunes = repository.findAllAsync().memoize().indefinitely(); - - // once we get a result for fortunes, - // we know its size and can thus query the right number of quotes - var quotes = fortunes.onItem().transformToUni(list -> getQuotesAsync(list.size())); - - // we now need to combine the two reactive streams - // before returning the result to the user - return Uni.combine().all().unis(fortunes, quotes).asTuple().onItem().transform(tuple -> { - var todoList = tuple.getItem1(); - //can await it since it is already resolved - var quotesList = tuple.getItem2(); - for(int i=0; i < todoList.size(); i ++){ - todoList.get(i).title += " - " + quotesList.get(i); - } - return todoList; - }); - } +== Detect pinned thread in tests - @GET - @RunOnVirtualThread - @Path("/quoted-virtual-thread") - public List getAllQuotedVirtualThread() { - //we get the list of fortunes - var fortunes = repository.findAllAsyncAndAwait(); - - //we get the list of quotes - var quotes = getQuotesAsync(fortunes.size()).await().indefinitely(); - - //we append each quote to each fortune - for(int i=0; i < fortunes.size(); i ++){ - fortunes.get(i).title += " - " + quotes.get(i); - } - return fortunes; - } -} +We recommend to use the following configuration when running tests in application using virtual threads. +If would not fail the tests, but at least dump start traces if the code pins the carrier thread: + +[source, xml] +---- + + maven-surefire-plugin + ${surefire-plugin.version} + + + org.jboss.logmanager.LogManager + ${maven.home} + + --enable-preview -Djdk.tracePinnedThreads + + ---- -== Pinning cases -The notion of "cheap blocking" might not always be true: in certain occasions a virtual thread might "pin" its carrier -(the platform thread it is mounted upon). -In this situation, the platform thread is blocked exactly as it would have been in a typical blocking scenario. +NOTE: The `--enable-preview` flag is not necessary with Java 21. -According to link:{vthreadjep}[JEP 425] this can happen in two situations: +== Run application using virtual threads -- when a virtual thread performs a blocking operation inside a `synchronized` block or method -- when it executes a blocking operation inside a native method or a foreign function +Prior to Java 21, virtual threads are still an experimental feature, you need to start your application with the `--enable-preview` flag: -It can be fairly easy to avoid these situations in our own code, but it is hard to verify every dependency we use. -Typically, while experimenting with virtual-threads, we realized that using the link:{pgsql-driver}[postgresql-JDBC driver] -results in frequent pinning. +[source, bash] +---- +java --enable-preview -jar target/quarkus-app/quarkus-run.jar +---- -=== The JDBC problem -Our experiments so far show that when a virtual thread queries a database using the JDBC driver, it will pin its carrier -thread during the entire operation. +== Build containers for application using virtual threads -Let's show the code of the `findAllBlocking()` method we used in the first example +When running your application in JVM mode (so not compiled into native, for native check xref:native[the dedicated section]), you can follow the xref:./container-image.adoc[containerization guide] to build a container. -[source, java] ----- -//import ... - -@ApplicationScoped -public class FortuneRepository { - // ... - - public List findAllBlocking() { - List fortunes = new ArrayList<>(); - Connection conn = null; - try { - conn = db.getJdbcConnection(); - var preparedStatement = conn.prepareStatement(SELECT_ALL); - ResultSet rs = preparedStatement.executeQuery(); - while (rs.next()) { - fortunes.add(create(rs)); - } - rs.close(); - preparedStatement.close(); - } catch (SQLException e) { - logger.warn("Unable to retrieve fortunes from the database", e); - } finally { - close(conn); - } - return fortunes; - } +In this section, we use JIB to build the container. +Refer to the xref:./container-image.adoc[containerization guide] to learn more about the alternatives. - //... -} +To containerize your Quarkus application that use `@RunOnVirtualThread`, add the following properties in your `application.properties`: + +[source, properties] ---- +quarkus.container-image.build=true +quarkus.container-image.group= +quarkus.container-image.name= +quarkus.jib.base-jvm-image=eclipse-temurin:20.0.1_9-jre-ubi9-minimal <1> +quarkus.jib.platforms=linux/amd64,linux/arm64 <2> +quarkus.jib.jvm-arguments=--enable-preview <3> +---- +<1> Make sure you use a base image supporting virtual threads. Here we use an image providing Java 20. +<2> Select the target architecture. You can select more than one to build multi-archs images. +<3> Don't forget to use the `--enable-preview` flag if you are not using Java 21+. -The actual query happens at `ResultSet rs = preparedStatement.executeQuery();`, here is how it is implemented in the -postgresql-jdbc driver 42.5.0: +Then, build your container as you would do usually. +For example, if you are using Maven, run: -[source, java] +[source, bash] ---- -class PgPreparedStatement extends PgStatement implements PreparedStatement { - // ... - - /* - * A Prepared SQL query is executed and its ResultSet is returned - * - * @return a ResultSet that contains the data produced by the * query - never null - * - * @exception SQLException if a database access error occurs - */ - @Override - public ResultSet executeQuery() throws SQLException { - synchronized (this) { - if (!executeWithFlags(0)) { - throw new PSQLException(GT.tr("No results were returned by the query."), PSQLState.NO_DATA); - } - return getSingleResultSet(); - } - } - - // ... -} +mvn package ---- -This `synchronized` block is the culprit. -Replacing it with a lock is a good solution, but it won't be enough: `synchronized` blocks are also used in `executeWithFlags(int flag)`. -A systematic review of the postgresql-jdbc driver is necessary to make sure that it is compliant with virtual threads. +[[native]] +== Compiling Quarkus application using virtual threads into native executable -NOTE: Recent versions of the Postgresql driver and MariaDB driver remove the pinning. -However, pinning may still happen because of the rest of the stack. +=== Using a local GraalVM installation -=== Reactive drivers at the rescue -The vertx-sql-client is a reactive client, hence it is not supposed to block while waiting for the completion of a -transaction with the database. -However, when using the link:{mutiny-vertx-sql}[smallrye-mutiny-vertx-sqlclient] it is possible to use a variant method -that will await for the completion of the transaction, mimicking a blocking behaviour. +To compile a Quarkus applications leveraging `@RunOnVirtualThreads` into native executable, you must be sure to use a GraalVM / Mandrel `native-image` supporting virtual threads, so providing at least Java 19+. -Below is the `FortuneRepository` except the blocking we've seen earlier has been replaced by reactive methods. +Then, until Java 21, you need to add the following property to your `application.properties` file: -[source, java] +[source, properties] +---- +quarkus.native.additional-build-args=--enable-preview ---- -//import ... -@ApplicationScoped -public class FortuneRepository { - // ... +Build the native executable as indicated on xref:./building-native-image.adoc[the native compilation guide]. +For example, with Maven, run: + +[source, bash] +---- +mvn package -Dnative +---- - public Uni> findAllAsync() { - return db.getPool() - .preparedQuery(SELECT_ALL).execute() - .map(this::createListOfFortunes); +=== Using an in-container build - } +In-container build allows building Linux 64 executables by using a `native-image` compiler running in a container. +It avoids having to install `native-image` on your machine, and also allows configuring the GraalVM version you need. +Note that, to use in-container build, you must have Docker or Podman installed on your machine. - public List findAllAsyncAndAwait() { - var rows = db.getPool().preparedQuery(SELECT_ALL) - .executeAndAwait(); - return createListOfFortunes(rows); - } +Then, add to your `application.properties` file: - //... -} +[source, properties] ---- +quarkus.native.additional-build-args=--enable-preview <1> -Contrary to the link:{pgsql-driver}[postgresql-jdbc driver], no `synchronized` block is used where it shouldn't be, and -the `await` behaviour is implemented using locks and latches that won't cause pinning. - -Using the synchronous methods of the link:{mutiny-vertx-sql}[smallrye-mutiny-vertx-sqlclient] along with virtual threads -will allow you to use the synchronous blocking style, avoid pinning the carrier thread, and get performance close to a pure -reactive implementation. +# In-container build to get a linux 64 executable +quarkus.native.container-build=true <2> +quarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-20 <3> +---- +<1> The `enable-preview` flag in only necessary until Java 21. +<2> Enables the in-container build +<3> The builder container to use. Make sure it supports virtual threads -== A point about performance +[IMPORTANT] +.From ARM/64 to AMD/64 +==== +If you are using a Mac M1 or M2 (using an ARM64 CPU), you need to be aware that the native executable you will get using an in-container build will be a Linux executable, but using your host (ARM 64) architecture. +You can use emulation to force the architecture when using Docker with the following property: -Our experiments seem to indicate that Quarkus with virtual threads will scale better than Quarkus blocking (offloading -the computation on a pool of platform worker threads) but not as well as Quarkus reactive. -The memory consumption especially might be an issue: if your system needs to keep its memory footprint low we would -advise you stick to using reactive constructs. +[source, properties] +---- +quarkus.native.container-runtime-options=--platform=linux/amd64 +---- -This degradation of performance doesn't seem to come from virtual threads themselves but from the interactions between -Vert.x/Netty (Quarkus underlying reactive engine) and the virtual threads. -This was illustrated in the issue that we will now describe. +Be aware that it increases the compilation time... a lot (>10 minutes). +==== -=== The Netty problem -For JSON serialization, Netty uses their custom implementation of thread locals, `FastThreadLocal` to store buffers. -When using virtual threads in quarkus, the number of virtual threads simultaneously living in the service is directly -related to the incoming traffic. -It is possible to get hundreds of thousands, if not millions, of them. +=== Containerize native applications using virtual threads -If they need to serialize some data to JSON they will end up creating as many instances of `FastThreadLocal`, resulting -on a massive memory consumption as well as exacerbated pressure on the garbage collector. -This will eventually affect the performance of the application and inhibit its scalability. +To build a container running a Quarkus application using virtual threads compiled into a native executable, you must +make sure you have a Linux/AMD64 executable (or ARM64 if you are targeting ARM machines). -This is a perfect example of the mismatch between the reactive stack and the virtual threads. -The fundamental hypothesis are completely different and result in different optimizations. -Netty expects a system using few event-loops (as many event-loops as CPU cores by default in Quarkus), but it gets hundreds -of thousands of threads. -You can refer to link:https://mail.openjdk.org/pipermail/loom-dev/2022-July/004844.html[this mail] to get more information -on how we envision our future with virtual threads. +Make sure your `application.properties` contains the configuration explained in xref:native[the native compilation section]. -=== Concerning dev mode -If you want to use quarkus with the dev mode, it won't be possible to manually specify the flags we mentioned along this guide. -Instead, you want to specify them all in the configuration of the `quarkus-maven-plugin` as presented below. +Then, build your container as you would do usually. +For example, if you are using Maven, run: -[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"] -.pom.xml +[source, bash] ---- - - io.quarkus - quarkus-maven-plugin - ${quarkus.version} - - - - build - - - - - - 20 - 20 - - --enable-preview - - --enable-preview --add-opens java.base/java.lang=ALL-UNNAMED - - - +mvn package -Dnative ---- -If you don't want to specify the opening of the `java.lang` module in your pom.xml file, you can also specify it as an argument -when you start the dev mode. +NOTE: If you ever want to build a native container image and already have an existing native image you can set `-Dquarkus.native.reuse-existing=true` and the native image build will not be re-run. -The configuration of the quarkus-maven-plugin will be simpler: +== Use the duplicated context in virtual threads -[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"] -.pom.xml ----- - - 20 - 20 - - --enable-preview - - --enable-preview - ----- +Methods annotated with `@RunOnVirtualThread` inherit from the original duplicated context (See the xref:duplicated-context.adoc[duplicated context reference guide] for details). +So, the data written in the duplicated context (and the request scope, as the request scoped is stored in the duplicated context) by filters and interceptors are available during the method execution (even if the filters and interceptors are not run on the virtual thread). -And the command will become: +However, thread locals are not propagated. -[source, bash] ----- -mvn quarkus:dev -Dopen-lang-package ----- +== Additional references + +- https://dl.acm.org/doi/10.1145/3583678.3596895[Considerations for integrating virtual threads in a Java framework: a Quarkus example in a resource-constrained environment] \ No newline at end of file