composeUp using captureContainersOutputToFiles blocks gradle from completing

[JWood48]

Using composeUp and configuring a output log directory like:

captureContainersOutputToFiles = "..."

Prevents gradle from completing and gradle hangs with the following output:

BUILD SUCCESSFUL in 23s
1 actionable tasks: 1 executed
<-------------> 0% WAITING

IDLE
IDLE

I would expect the task to complete and the logs being written to the configured directory?

Or is it the gradle process that is piping the logs to the log directory so it must keep running?

(test using gradle 5.2.1)

[augi]

Could you please prepare a simple project that proves this behavior? Thanks!

[LukasWysocki]

Hit the same problem.
Repo with reproducer is here:
https://github.com/LukasWysocki/gradle-docker-compose-plugin-issue-188

This is docker-compose using busybox to run single command "sleep 42"

to test do:

git clone [email protected]:LukasWysocki/gradle-docker-compose-plugin-issue-188.git
cd gradle-docker-compose-plugin-issue-188
time gradle composeUp

The effect is:

lwysocki@lwy-thinkpad:~/high5/gradle-docker-compose-plugin-issue-188$ time gradle composeUp

> Task :composeUp
sleep uses an image, skipping
Will use 172.24.0.1 (network 844c3edbeb4315546c21b61404a6a645_gradle-docker-compose-plugin-issue-188__default) as host of sleep

BUILD SUCCESSFUL in 6s
1 actionable task: 1 executed

real    0m47,509s
user	0m12,744s
sys	0m0,704s

Notice BUILD SUCCESSFUL in 6s however time reports real 0m47,509s . Which is the ~42 seconds of sleep + ~6s of build time.

Forgot to mention 2 gradle versions tested:

gradle --version

------------------------------------------------------------
Gradle 4.10.2
------------------------------------------------------------

Build time:   2018-09-19 18:10:15 UTC
Revision:     b4d8d5d170bb4ba516e88d7fe5647e2323d791dd

Kotlin DSL:   1.0-rc-6
Kotlin:       1.2.61
Groovy:       2.4.15
Ant:          Apache Ant(TM) version 1.9.11 compiled on March 23 2018
JVM:          1.8.0_201 (Oracle Corporation 25.201-b09)
OS:           Linux 4.15.0-47-generic amd64

and:

./gradlew --version

------------------------------------------------------------
Gradle 4.9
------------------------------------------------------------

Build time:   2018-07-16 08:14:03 UTC
Revision:     efcf8c1cf533b03c70f394f270f46a174c738efc

Kotlin DSL:   0.18.4
Kotlin:       1.2.41
Groovy:       2.4.12
Ant:          Apache Ant(TM) version 1.9.11 compiled on March 23 2018
JVM:          1.8.0_201 (Oracle Corporation 25.201-b09)
OS:           Linux 4.15.0-47-generic amd64

[augi]

Thank you for your time you invested to reporting this issue! 👍

Unfortunately, I'm not able to reproduce the issue even with repository you prepared :( My output is this:

$ time ./gradlew composeUp

> Task :composeUp
sleep uses an image, skipping
Will use localhost as host of sleep

BUILD SUCCESSFUL in 6s
1 actionable task: 1 executed

real    0m7.404s
user    0m0.045s
sys     0m0.197s

------------------------------------------------------------
Gradle 5.3.1
------------------------------------------------------------

Build time:   2019-03-28 09:09:23 UTC
Revision:     f2fae6ba563cfb772c8bc35d31e43c59a5b620c3

Kotlin:       1.3.21
Groovy:       2.5.4
Ant:          Apache Ant(TM) version 1.9.13 compiled on July 10 2018
JVM:          11.0.1 (AdoptOpenJDK 11.0.1+13)
OS:           Windows 10 10.0 amd64

The only difference is that you are executing gradle command but I don't have Gradle locally installed so I'm using Gradle Wrapper.

The thread to read containers output is executed as background thread so it shouldn't block process shutdown.

My only idea is that you are using Gradle daemon - so could you please execute the same command with --no-daemon added?

[augi]

To fix this issue, we should forcibly close the daemon threads when Gradle execution finishes. Then using Gradle daemon, it's not the same as JVM shutdown. Unfortunately, I'm not aware of any Gradle API that would allow this.

On other hand, the current behavior is required for some use-cases, like people start using composeUp, do other things (outside of Gradle), and then stop the containers using composeDown (in separate Gradle execution).

I would like to add an option to change the behavior to support also your use-case anyway! If you know about a Gradle hook that would be executed after Gradle execution finishes, please let me know.

[augi]

Now, we are using a hook to interrupt the daemon threads after Gradle build finishes. It's released in 0.9.3.