Support running the same application multiple times in dev mode #22743

Postremus · 2022-01-07T21:29:55Z

Adds support for running the same application multiple times in dev mode on the same machine.
Problem was, that resources of webjars (like devui) where unpacked to %TEMP%/quarkus/applicationName/libraryName/libraryVersion.
On windows, this worked fine. %TEMP% points to a path inside the user directory (e.g. c:/users/martin/appdata/local/temp/).
On linux however, this caused AccessDeniedException, since %TEMP% points to a global temp directory, e.g. /temp.

This branch changes the location where webjars are unpacked to.
Now, every quarkus dev mode isntace creates a new temp directory, e.g. C:\Users\Martin\AppData\Local\Temp\quarkus-webjar10464518099488497426.
These temp directories are removed on shutdown.

The directory layout inside the temp directory is changed. It is now flatter, and the same as in the generated resource. e.g. quarkus-webjar10464518099488497426\META-INF\swagger-ui-files\index.html

I also deprecated io.quarkus.deployment.util.WebJarUtil (forRemoval) in favor of two new BuildItems - WebJarBuildItem and WebJarResultBuildItem.
WebJarBuildItem contains the instructions on which artifact should be unpacked and to which final destination (e.g. META-INF/swagger-ui-files). The webjar is either unpacked to disk (dev mode), or resources are generated (prod).
WebJarResultBuildItem contains the path where the webjar was unpacked to.

dev-ui, swagger-ui, smallrye-health and graphql already use these new builditems - in quarkus, no place uses the deprecated WebJarUtil anymore.

fixes #22515

Related zulip discussion: https://quarkusio.zulipchat.com/#narrow/stream/187038-dev/topic/Refactoring.20WebJarUtil

Postremus · 2022-01-07T21:32:48Z

.../swagger-ui/deployment/src/main/java/io/quarkus/swaggerui/deployment/SwaggerUiProcessor.java

-            Handler<RoutingContext> handler = recorder.handler(finalDestinationBuildItem.getSwaggerUiFinalDestination(),
-                    finalDestinationBuildItem.getSwaggerUiPath(),
+            String swaggerUiPath = nonApplicationRootPathBuildItem.resolvePath(swaggerUiConfig.path);
+            swaggerUiBuildProducer.produce(new SwaggerUiBuildItem(result.getFinalDestination(), swaggerUiPath));


SwaggerUiBuildItem is produced just to be on the safe side, no idea if anyone consumes it.

Maybe we can deprecate the build item as well?

Postremus · 2022-01-07T21:33:08Z

.../deployment/src/main/java/io/quarkus/smallrye/health/deployment/SmallRyeHealthProcessor.java

-                    smallRyeHealthBuildItem.getHealthUiPath(), runtimeConfig);
+            String healthUiPath = nonApplicationRootPathBuildItem.resolvePath(healthConfig.ui.rootPath);
+            smallryeHealthBuildProducer
+                    .produce(new SmallRyeHealthBuildItem(result.getFinalDestination(), healthUiPath));


SmallRyeHealthBuildItem is produced just to be on the safe side, no idea if anyone consumes it.

Maybe we can deprecate the build item as well?

quarkus-bot · 2022-01-09T01:20:34Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `f5ed857`

Status	Name	Step	Failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Gradle Tests - JDK 11 Windows #

- Failing: integration-tests/gradle

📦 integration-tests/gradle

✖ io.quarkus.gradle.devmode.ModuleWithParentDependencyDevModeTest.main line 14 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference, java.util.concurrent.atomic.AtomicReferencejava.lang.String, java.lang.Stringboolean was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:164)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

Postremus · 2022-01-10T06:52:02Z

This PR is ready for review - it contains everything I wanted to include for now.

Dev mode startup time worsens by about 30ms. According to profiling, this is because the temp directory is now deleted on shutdown, and webjar resources need to be copied again.

I plan to open a follow up PR once this one is merged which will implement #22525 for swagger-ui, smallrye-health, graphql.
Should be enough to counteract these 30ms.

Postremus · 2022-01-10T06:53:42Z

...eployment/src/main/java/io/quarkus/smallrye/graphql/deployment/SmallRyeGraphQLProcessor.java

-            Handler<RoutingContext> handler = recorder.uiHandler(smallRyeGraphQLBuildItem.getGraphqlUiFinalDestination(),
-                    smallRyeGraphQLBuildItem.getGraphqlUiPath(), runtimeConfig);
+            String graphQLUiPath = nonApplicationRootPathBuildItem.resolvePath(graphQLConfig.ui.rootPath);
+            smallRyeGraphQLBuildProducer


SmallRyeGraphQLBuildItem is produced just to be on the safe side, no idea if anyone consumes it.

Maybe we can deprecate the build item as well?

aloubyansky · 2022-01-12T12:09:58Z

.../vertx-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarUtil.java

+
+        ClassLoader classLoader = WebJarUtil.class.getClassLoader();
+
+        for (Path p : resourcesArtifact.getResolvedPaths()) {


We should be moving away from using resolved path to explore the content of a resolved dependency switching to PathTree userApplication.getContentTree(). Then you walk the tree or process a specific path.

aloubyansky · 2022-01-12T12:12:07Z

.../vertx-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarUtil.java

+            String artifactId) {
+        for (ResolvedDependency dep : curateOutcomeBuildItem.getApplicationModel().getDependencies()) {
+            if (dep.getArtifactId().equals(artifactId)
+                    && dep.getGroupId().equals(groupId)) {


Keep in mind, it may happen there could be multiple artifacts on the classpath with the same G:A but different classifiers.

We now also compare the classifier in this place.
I saw no other way to compare a GACT to the GACTV of the ResolvedDependency.

aloubyansky · 2022-01-12T21:10:30Z

.../vertx-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarUtil.java

+
+        resourcesArtifact.getContentTree().accept(webJar.getRoot(), new Consumer<PathVisit>() {
+            @Override
+            public void accept(PathVisit pathVisit) {


Theoretically, the passed in pathVisit could be null if the content tree does not contain the path (webJar.getRoot), not sure if that could happen in this specific case though. I thought it would be up to the impl of the consumer whether to throw an error or simply return in such a case.

aloubyansky · 2022-01-12T21:15:07Z

.../vertx-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarUtil.java

+        for (ResolvedDependency dep : curateOutcomeBuildItem.getApplicationModel().getDependencies()) {
+            if (dep.getArtifactId().equals(artifactKey.getArtifactId())
+                    && dep.getGroupId().equals(artifactKey.getGroupId())
+                    && dep.getClassifier().equals(artifactKey.getClassifier())) {


That or dep.getKey().equals(artifactKey)

I thought about it, too, but I don't see how .equals can return true.
ResolvedDependency.getKey() returns an instance of GACTV.

We only pass a GACT in here, since the version of the webjar is not static.
E.g. io.quarkus: quarkus-vertx-http-deployment is one of it.

I thought about it, too, but I don't see how .equals can return true. ResolvedDependency.getKey() returns an instance of GACTV.

No, it returns an ArtifactKey, which is GACT.

yeah, you are obviously right. I guess I got confused, because ResolvedArtifactDependency extends GACTV. :D

Postremus · 2022-01-15T17:58:14Z

I cleaned this up a bit further, and implemented following additonal changes / fixes:

onlyCopyNonArtifactFiles now has no effect in prod builds, since all files are needed anyway.
insertVariables is now also a simple WebJarResourcesFilter
CombinedWebJarResoucesFilter allows to combine multiple filters into one. This allows to propagate the changed flag through all filters. For example, SwaggerUIProcessor generated a changed index.html file. With onlyCopyNonArtifactFiles=true, this file was not copied to disk however, since no change was detected.

quarkus-bot · 2022-01-15T22:20:30Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `2455ecd`

Status	Name	Step	Failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Gradle Tests - JDK 11 Windows #

- Failing: integration-tests/gradle

📦 integration-tests/gradle

✖ io.quarkus.gradle.devmode.QuarkusDevDependencyDevModeTest.main line 14 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference, java.util.concurrent.atomic.AtomicReferencejava.lang.String, java.lang.Stringboolean was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:164)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

gsmet

Finally had the time to look at this one. I also migrated the Quarkus GitHub App to it to make sure things would be smooth.

Things are indeed far better with this setup and I was able to remove quite some boilerplate that was very ad hoc.

So it's a big +1 from me but... I added a few suggestions. They are open to discussion so feel free to push back :).

gsmet · 2022-01-17T13:06:35Z

.../deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarResultBuildItem.java

+import io.quarkus.maven.dependency.ArtifactKey;
+import io.quarkus.maven.dependency.ResolvedDependency;
+
+public final class WebJarResultBuildItem extends SimpleBuildItem {


Could we have some class-level javadoc here?

gsmet · 2022-01-17T13:21:39Z

...ment/src/main/java/io/quarkus/vertx/http/deployment/devmode/console/DevConsoleProcessor.java

+    private static final GACT DEVCONSOLE_WEBJAR_ARTIFACT_KEY = new GACT("io.quarkus", "quarkus-vertx-http-deployment", null,
+            "jar");
+    private static final String DEVCONSOLE_WEBJAR_STATIC_RESOURCES_PATH = "dev-static/";
+    private static final String DEVCONSOLE_FINAL_DESTINATION = "META-INF/dev-ui-files";


Why do we need this now? Couldn't it be generated from the artifact id/version + resource path? That would simplify things a bit.
At least I don't think we should make it mandatory.

I removed this option completly. Automaticly generating the final destination also prevents clashes between different webjars, which is quite nice.

gsmet · 2022-01-17T13:31:38Z

.../deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarResultBuildItem.java

+import io.quarkus.maven.dependency.ArtifactKey;
+import io.quarkus.maven.dependency.ResolvedDependency;
+
+public final class WebJarResultBuildItem extends SimpleBuildItem {


I think I would call it WebJarResultsBuildItem plural. You have several and it would be less confusing.

gsmet · 2022-01-17T13:32:47Z

...x-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarBuildItem.java

+        this(artifactKey, webjarRoot, true, false, finalDestination, filter);
+    }
+
+    public WebJarBuildItem(GACT webJarKey, String webjarRoot, boolean useDefaultQuarkusBranding,


I think it would be handy to have versions of both constructors without the filter as it's purely optional.

gsmet · 2022-01-17T13:38:20Z

core/deployment/src/main/java/io/quarkus/deployment/util/WebJarUtil.java

@@ -44,7 +44,10 @@

 /**
 * Utility for Web resource related operations
+ * 
+ * @deprecated Use WebJarBuildItem and WebJarResultBuildItem instead.


If you rename WebJarResultBuildItem, make sure this is done everywhere including here.

gsmet · 2022-01-17T13:39:41Z

...x-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarBuildItem.java

+    private final boolean onlyCopyNonArtifactFiles;
+
+    /**
+     * Defines whether quarkus can override resources of the webjar with quarkus internal files.


Suggested change

* Defines whether quarkus can override resources of the webjar with quarkus internal files.

* Defines whether Quarkus can override resources of the webjar with Quarkus internal files.

gsmet · 2022-01-17T13:41:07Z

...x-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarBuildItem.java

+        this(artifactKey, webjarRoot, true, false, finalDestination, filter);
+    }
+
+    public WebJarBuildItem(GACT webJarKey, String webjarRoot, boolean useDefaultQuarkusBranding,


That being said, I wonder if we should use a Builder approach for this one?

gsmet · 2022-01-17T13:41:28Z

...x-http/deployment/src/main/java/io/quarkus/vertx/http/deployment/webjar/WebJarBuildItem.java

+        this(artifactKey, webjarRoot, true, false, finalDestination, filter);
+    }
+
+    public WebJarBuildItem(GACT webJarKey, String webjarRoot, boolean useDefaultQuarkusBranding,


Maybe rename webJarKey to artifactKey for consistency?

Postremus · 2022-01-17T21:20:58Z

Thank you for the review @gsmet.
I implemented every change.

quarkus-bot · 2022-01-18T01:38:30Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `f9f1334`

Status	Name	Step	Failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Failures	Logs	Raw logs
✖	JVM Tests - JDK 11 Windows	`Build`	Failures	Logs	Raw logs
✖	JVM Tests - JDK 17	`Build`	Failures	Logs	Raw logs
✖	Native Tests - Amazon	`Build`	Failures	Logs	Raw logs
✖	Native Tests - Main	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ JVM Tests - JDK 11 #

- Failing: integration-tests/spring-web

📦 integration-tests/spring-web

✖ io.quarkus.it.spring.web.openapi.SwaggerAndOpenAPIWithCommonPrefixPMT.shouldWorkEvenWithCommonPrefix line 28 - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code <200> but was <404>.

⚙️ JVM Tests - JDK 11 Windows #

- Failing: integration-tests/spring-web

📦 integration-tests/spring-web

✖ io.quarkus.it.spring.web.openapi.SwaggerAndOpenAPIWithCommonPrefixPMT.shouldWorkEvenWithCommonPrefix line 28 - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code <200> but was <404>.

⚙️ JVM Tests - JDK 17 #

- Failing: integration-tests/spring-web

📦 integration-tests/spring-web

✖ io.quarkus.it.spring.web.openapi.SwaggerAndOpenAPIWithCommonPrefixPMT.shouldWorkEvenWithCommonPrefix line 28 - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code <200> but was <404>.

⚙️ Native Tests - Amazon #

- Failing: integration-tests/amazon-lambda-http

📦 integration-tests/amazon-lambda-http

✖ io.quarkus.it.amazon.lambda.AmazonLambdaSimpleIT.testSwaggerUi - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code <200> but was <404>.

⚙️ Native Tests - Main #

- Failing: integration-tests/main

📦 integration-tests/main

✖ io.quarkus.it.main.SwaggerUIITCase.testSwaggerUi - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Response body doesn't match expectation.

Postremus · 2022-01-18T05:29:38Z

test failures look related, will have to take a closer look.

gsmet · 2022-01-18T10:44:54Z

@Postremus just a note that the cutoff for 2.7.0.CR1 is tonight and I would prefer having this one in CR1 given how structural it is.

Postremus · 2022-01-18T16:32:58Z

Should be fixed now, lets see what CI has to say.
The generated finalDestination ended with a slash, which wasn't expected by the SwaggerUIHandler / StaticFileHandlerImpl.

gsmet · 2022-01-18T21:48:59Z

And all green! Thanks for your efforts on this @Postremus, it's much cleaner!

Postremus requested review from phillip-kruger and gsmet January 7, 2022 21:30

quarkus-bot bot added area/core area/health area/smallrye area/swagger-ui area/vertx labels Jan 7, 2022

Postremus commented Jan 7, 2022

View reviewed changes

Postremus force-pushed the bugs/22515 branch 2 times, most recently from 8c17e36 to f5ed857 Compare January 8, 2022 21:18

quarkus-bot bot added the area/graphql label Jan 8, 2022

Postremus force-pushed the bugs/22515 branch from f5ed857 to 411aa3a Compare January 9, 2022 20:18

Postremus commented Jan 10, 2022

View reviewed changes

Postremus force-pushed the bugs/22515 branch from 411aa3a to bb3c56c Compare January 10, 2022 18:08

aloubyansky reviewed Jan 12, 2022

View reviewed changes

gsmet self-assigned this Jan 12, 2022

Postremus force-pushed the bugs/22515 branch from bb3c56c to e52fd02 Compare January 12, 2022 20:58

aloubyansky reviewed Jan 12, 2022

View reviewed changes

Postremus force-pushed the bugs/22515 branch from e52fd02 to 2f1b2df Compare January 13, 2022 19:57

Postremus marked this pull request as draft January 15, 2022 15:48

Postremus force-pushed the bugs/22515 branch from 2f1b2df to 2455ecd Compare January 15, 2022 17:49

Postremus marked this pull request as ready for review January 15, 2022 17:58

gsmet requested changes Jan 17, 2022

View reviewed changes

Postremus force-pushed the bugs/22515 branch from 2455ecd to f9f1334 Compare January 17, 2022 21:14

Support running the same application multiple times in dev mode

2996f38

Postremus force-pushed the bugs/22515 branch from f9f1334 to 2996f38 Compare January 18, 2022 16:32

gsmet approved these changes Jan 18, 2022

View reviewed changes

gsmet added this to the 2.7 - main milestone Jan 18, 2022

gsmet added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Jan 18, 2022

gsmet merged commit a039ce4 into quarkusio:main Jan 18, 2022

quarkus-bot bot removed the triage/waiting-for-ci Ready to merge when CI successfully finishes label Jan 18, 2022

Postremus deleted the bugs/22515 branch January 19, 2022 06:33


		ClassLoader classLoader = WebJarUtil.class.getClassLoader();

		for (Path p : resourcesArtifact.getResolvedPaths()) {

	* Defines whether quarkus can override resources of the webjar with quarkus internal files.
	* Defines whether Quarkus can override resources of the webjar with Quarkus internal files.

Support running the same application multiple times in dev mode #22743

Support running the same application multiple times in dev mode #22743

Conversation

Postremus commented Jan 7, 2022 • edited Loading

Postremus Jan 7, 2022 • edited Loading

Choose a reason for hiding this comment

Postremus Jan 7, 2022 • edited Loading

Choose a reason for hiding this comment

quarkus-bot bot commented Jan 9, 2022 • edited Loading

Failing Jobs - Building f5ed857

Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

Postremus commented Jan 10, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Postremus commented Jan 15, 2022

quarkus-bot bot commented Jan 15, 2022 • edited Loading

Failing Jobs - Building 2455ecd

Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

gsmet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Postremus commented Jan 17, 2022

quarkus-bot bot commented Jan 18, 2022 • edited Loading

Failing Jobs - Building f9f1334

Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/spring-web

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/spring-web

⚙️ JVM Tests - JDK 17 #

📦 integration-tests/spring-web

⚙️ Native Tests - Amazon #

📦 integration-tests/amazon-lambda-http

⚙️ Native Tests - Main #

📦 integration-tests/main

Postremus commented Jan 18, 2022 • edited Loading

gsmet commented Jan 18, 2022

Postremus commented Jan 18, 2022 • edited Loading

gsmet commented Jan 18, 2022

Postremus commented Jan 7, 2022 •

edited

Loading

Postremus Jan 7, 2022 •

edited

Loading

Postremus Jan 7, 2022 •

edited

Loading

quarkus-bot bot commented Jan 9, 2022 •

edited

Loading

Failing Jobs - Building `f5ed857`

quarkus-bot bot commented Jan 15, 2022 •

edited

Loading

Failing Jobs - Building `2455ecd`

quarkus-bot bot commented Jan 18, 2022 •

edited

Loading

Failing Jobs - Building `f9f1334`

Postremus commented Jan 18, 2022 •

edited

Loading

Postremus commented Jan 18, 2022 •

edited

Loading