Using quarkusDev with gradle leaves process running

[briankahoot]

Describe the bug

When running quarkus in dev mode, using ./gradlew quarkusDev, the service is started correctly but when gradle is closed (by pressing Ctrl+C) the process is left running. Subsequent attempts to run in dev mode result in a port conflict for the debug port and an increasing number of processes left behind.

Expected behavior

When you terminate the gradle build then the associated process running the quarkus jar should also be killed.

Actual behavior

The java process running the quarkus jar is left running.

To Reproduce

When creating a new gradle project using the maven template this issue does not initially occur and when you stop the gradle build the java process is also stopped. However adding certain extensions such as io.quarkus:quarkus-micrometer-registry-prometheus causes this problem to occur, as long as you have metrics enabled.

Link to a small reproducer (preferably a Maven project if the issue is not Gradle-specific).
https://github.com/briancullen/quarkus-reproducer

Steps to reproduce the behavior:

1. Clone the git repo
2. Run ./gradlew quarkusDev
3. Press Ctrl+C to stop the build process.

Environment (please complete the following information):

Output of uname -a or ver

Linux xxxxx 5.8.0-48-generic #54~20.04.1-Ubuntu SMP Sat Mar 20 13:40:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.8+10)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.21.0, JRE 11 Linux amd64-64-Bit Compressed References 20200715_697 (JIT enabled, AOT enabled)
OpenJ9 - 34cf4c075
OMR - 113e54219
JCL - 95bb504fbb based on jdk-11.0.8+10)

GraalVM version (if different from Java)

Quarkus version or git rev

1.13.0.Final

Build tool (ie. output of mvnw --version or gradlew --version)

---

Gradle 6.8.3

Build time: 2021-02-22 16:13:28 UTC
Revision: 9e26b4a9ebb910eaa1b8da8ff8575e514bc61c78

Kotlin: 1.4.20
Groovy: 2.5.12
Ant: Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM: 11.0.8 (Eclipse OpenJ9 openj9-0.21.0)
OS: Linux 5.8.0-48-generic amd64

[quarkus-bot]

/cc @evanchooly, @glefloch, @quarkusio/devtools

[jc]

I've been seeing similar behavior using kotlin and maven.

Running quarkus:dev hitting Ctrl^C doesn't terminate the java process, I have to kill manually. I haven't had time to generate a reproducible test case but a thread dump shows that the Quarkus Shutdown Thread is stuck in waiting on a CountDownLatch: at io.quarkus.vertx.core.runtime.VertxCoreRecorder.shutdownDevMode(VertxCoreRecorder.java:166)

[glefloch]

I tried to reproduce the problem, but I was not able to. Do you have a sample project that reproduce the issue? @briankahoot, the sample project does not contains any metrics.

[jaikiran]

Looking at the code in VertxRecorder there's a possibility for this to block forever. I've created a PR with a potential fix #16253 (I don't have an easy way to verify that it fixes your issue).

[mkouba]

@jaikiran Hm, what makes you think that this reproducer can be fixed with your PR? The reproducer does not seem to depend on the quarkus-vertx extension and I see no method annotated with @ConsumeEvent there...

[jaikiran]

Hello @mkouba,

The change in my PR was (supposed to be) motivated by this comment:

I haven't had time to generate a reproducible test case but a thread dump shows that the Quarkus Shutdown Thread is stuck in waiting on a CountDownLatch: at io.quarkus.vertx.core.runtime.VertxCoreRecorder.shutdownDevMode(VertxCoreRecorder.java:166)

But now that you asked me this question, I checked again and the PR I opened, fixes this in a different recorder and not VertxCoreRecorder. So yes, I need to unlink that PR from this issue. I'll do that now.

As for the VertxCoreRecorder, its usage of the latch seems fine to me. It looks like the handler itself doesn't get notified (perhaps close() doesn't complete?). I haven't been able to reproduce this issue so far.

[mkouba]

But now that you asked me this question, I checked again and the PR I opened, fixes this in a different recorder and not VertxCoreRecorder. So yes, I need to unlink that PR from this issue. I'll do that now.

+1

As for the VertxCoreRecorder, its usage of the latch seems fine to me. It looks like the handler itself doesn't get notified (perhaps close() doesn't complete?). I haven't been able to reproduce this issue so far.

Indeed, the usage of the CountDownLatch in the VertxCoreRecorder.shutdownDevMode() looks OK.

[jc]

I can reproduce this using the kafka-streams-quickstart, adding micrometer-registry-prometheus to either producer or aggregator causes quarkus:dev to hang on termination.

mvn quarkus:add-extension -Dextensions="micrometer-registry-prometheus"

[mkouba]

CC @ebullient

[ebullient]

@mkouba .. this should have been fixed with our latest (joint) changes last week (re #16130)
PRs #16140 and #16164

Yours (the first) made it into 1.13.1.Final, so we should start w/ retrying against that.

[jc]

This is no longer an issue in 1.13.1.Final. Thanks!