Continuous testing #16251
Continuous testing #16251
Conversation
quarkus-bot
bot
added
area/config
area/core
area/dependencies
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
1233c8a to
475dcaa
Compare
|
Cool! I'll start looking into the magic! |
stuartwdouglas
force-pushed
the
continuous-testing
branch
3 times, most recently
from
7242b5e to
da46d00
Compare
core/deployment/src/main/java/io/quarkus/deployment/TestConfig.java
Outdated
Show resolved
Hide resolved
core/deployment/src/main/java/io/quarkus/deployment/TestConfig.java
Outdated
Show resolved
Hide resolved
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
368dccc to
ae0668b
Compare
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
ae0668b to
f275b7f
Compare
stuartwdouglas
added
the
triage/qe?
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
f275b7f to
39c8e79
Compare
|
This workflow status is outdated as a new workflow run has been triggered. Failing Jobs - Building 39c8e79
Test Failures⚙️ Maven Tests - JDK 11 #📦 integration-tests/maven✖ ✖ ✖ ✖ ✖ ✖ ⚙️ MicroProfile TCKs Tests #📦 resteasy-reactive-testsuite/tests✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ |
|
This workflow status is outdated as a new workflow run has been triggered. Failing Jobs - Building f275b7f
Test Failures⚙️ Gradle Tests - JDK 11 Linux #📦 integration-tests/gradle/build/resources/test/test-fixtures-module✖ 📦 integration-tests/gradle✖ ⚙️ Gradle Tests - JDK 11 Windows #📦 integration-tests/gradle/build/resources/test/test-fixtures-module✖ 📦 integration-tests/gradle✖ ✖ ⚙️ JVM Tests - JDK 15 #📦 extensions/elasticsearch-rest-client/deployment✖ 📦 extensions/vertx-http/deployment✖ ⚙️ JVM Tests - JDK 8 #📦 extensions/elasticsearch-rest-client/deployment✖ ⚙️ Maven Tests - JDK 11 #📦 integration-tests/maven✖ ✖ ✖ ✖ ✖ ✖ ⚙️ Maven Tests - JDK 11 Windows #📦 integration-tests/maven✖ ✖ ✖ ✖ ✖ ✖ ⚙️ MicroProfile TCKs Tests #📦 resteasy-reactive-testsuite/tests✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ ✖ |
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
39c8e79 to
811030b
Compare
|
I need the CI bandwidth, I'll restart as soon as I have the 1.13.2 backport PR passing. |
|
This workflow status is outdated as a new workflow run has been triggered. Failing Jobs - Building 811030b
|
gsmet
force-pushed
the
continuous-testing
branch
from
811030b to
a7cdf17
Compare
|
Rebased to get the latest CI fixes and retrigger CI. |
|
I have addressed the comments by @aloubyansky , it was green beforehand and the changes were minor so hopefully it stays that way. |
|
This workflow status is outdated as a new workflow run has been triggered. Failing Jobs - Building faed76a
|
| ClassLoader old = Thread.currentThread().getContextClassLoader(); | ||
| try { | ||
|
|
||
| ClassLoader tcl = testApplication.createDeploymentClassLoader(); |
There was a problem hiding this comment.
It doesn't look like this deployment CL is closed, is it? May lead to jars not being closed and locking issues on windows.
|
|
||
| public synchronized void abort() { | ||
| for (TestRunListener listener : listeners) { | ||
| listener.runAborted(); |
There was a problem hiding this comment.
Should this be wrapped in try-catch to make sure all the listeners are notified and the code after the loop gets executed?
| appPropertiesExcludeTags = excludeTags; | ||
| appPropertiesIncludePattern = includePattern; | ||
| appPropertiesExcludePattern = excludePattern; | ||
| } |
There was a problem hiding this comment.
A minor one, you could break the loop once the application.properties has been found.
| .addAdditionalApplicationArchive(new AdditionalDependency( | ||
| Paths.get(context.getApplicationRoot().getTest().get().getClassesPath()), true, | ||
| true)) | ||
| .build() |
There was a problem hiding this comment.
Not necessarily for this PR, I am wondering whether we could pass the AppModel instance to the test app instead of re-resolving it. That could save some time.
There was a problem hiding this comment.
This is a TEST appmodel, but the outer one is DEV. I want to talk to you about this though, there is a lot of stuff that could be cleaned up here.
| addSubstitutedProject(dep, includedBuild.getProjectDir()); | ||
| if ("test-fixtures".equals(a.getClassifier())) { | ||
| //TODO: test-fixtures are broken under the new ClassLoading model | ||
| dep.addPath(a.getFile()); |
| Optional<TestSupport> ts = TestSupport.instance(); | ||
| if (!ts.isPresent() || launchModeBuildItem.getDevModeType().orElse(null) != DevModeType.LOCAL) { | ||
| return null; | ||
| } |
There was a problem hiding this comment.
Looks like this if is present in 5 methods in this class, should it be extracted to something like isDevConsoleEnabled(LaunchModeBuildItem) or something?
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
6486e52 to
4e06c98
Compare
|
This workflow status is outdated as a new workflow run has been triggered. |
|
This workflow status is outdated as a new workflow run has been triggered. Failing Jobs - Building 4e06c98
Full information is available in the Build summary check run. Test Failures⚙️ Gradle Tests - JDK 11 Windows #📦 integration-tests/gradle/build/resources/test/inject-bean-from-test-config/application✖ 📦 integration-tests/gradle✖ ⚙️ JVM Tests - JDK 11 #📦 integration-tests/mailer✖ ⚙️ Native Tests - Data3 #📦 integration-tests/liquibase✖ |
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
4e06c98 to
4cfaf52
Compare
|
This workflow status is outdated as a new workflow run has been triggered. Failing Jobs - Building 4cfaf52
Full information is available in the Build summary check run. Test Failures⚙️ Maven Tests - JDK 11 #📦 integration-tests/maven✖ |
|
This workflow status is outdated as a new workflow run has been triggered. Failing Jobs - Building 4e6a12e
|
stuartwdouglas
force-pushed
the
continuous-testing
branch
2 times, most recently
from
929106a to
6c89234
Compare
|
This workflow status is outdated as a new workflow run has been triggered. 🚫 This workflow run has been cancelled. Failing Jobs - Building 929106a
|
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
6c89234 to
cb4ec0d
Compare
|
This workflow status is outdated as a new workflow run has been triggered. 🚫 This workflow run has been cancelled. Failing Jobs - Building 6c89234
|
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
cb4ec0d to
5fc1d79
Compare
|
This workflow status is outdated as a new workflow run has been triggered. 🚫 This workflow run has been cancelled. Failing Jobs - Building cb4ec0d
Full information is available in the Build summary check run. Test Failures⚙️ Gradle Tests - JDK 11 Windows #📦 integration-tests/gradle✖ |
|
I think I have figured out the intermittent Gradle failure. For some reason the test in question has the gradle jacoco plugin in the generated project, which is not used anywhere else. As we now resolve test dependencies this would now attempt to download this artifact, which may not always happen in 1 minute. |
|
This workflow status is outdated as a new workflow run has been triggered. Failing Jobs - Building 5fc1d79
|
This commit adds the ability to test Quarkus applications running in dev mode in a continous manner, only running tests that are affected by the changed code. To this is it changes the test ClassLoading model to work the same as the dev mode model, and adds the ability to launch a completly isolated test application inside the same JVM as an existing Quarkus app.
stuartwdouglas
force-pushed
the
continuous-testing
branch
from
5fc1d79 to
7ca3307
Compare
|
@gsmet all green now |
There was a problem hiding this comment.
OK, let's merge and iterate from there. This is a very significant and useful addition so I want to get this one in Alpha1 to get more eyeballs on it.
@stuartwdouglas I'll let you send me an email with a short description of the feature? Thanks!
This adds continuous testing support to Quarkus.