Reactive Messaging codestart #21068

vgallet · 2021-10-28T13:49:59Z

fix Reactive Messaging codestart (in java and kotlin) #20872
created MyReactiveMessagingApplication to demonstrates @Emitter, @Incoming and @Outgoing annotations.

quarkus-bot · 2021-10-28T13:50:01Z

Thanks for your pull request!

The title of your pull request does not follow our editorial rules. Could you have a look?

title should not contain an issue number (use Fix #1234 in the description instead)

This message is automatically generated by a bot.

vgallet · 2021-10-28T14:12:37Z

@ia3andy @ozangunalp

ia3andy · 2021-10-28T14:37:26Z

@vgallet
The code looks great.

You will need to add:

The dynamic connector app config depending on selected extensions (I can help if needed)
The tests
Linking the different connector extensions (in the metadata)
Kotlin (but that can be in another commit/PR)

vgallet · 2021-10-28T14:59:51Z

@ia3andy

The dynamic connector app config depending on selected extensions (I can help if needed)
Linking the different connector extensions (in the metadata)

My bad, I wasn't aware codestart could dynamic depending on the selected connector, I thought created a codestart per connector implementation. Can you show me some documentation or an example to do that ?

ozangunalp · 2021-10-28T20:03:15Z

For the dynamic config depending on the connector, I guess you'll need to do some Qute templating.
You can get inspiration from the Resteasy codestart : quarkus/independent-projects/tools/base-codestarts/src/main/resources/codestarts/quarkus/extension-codestarts/resteasy-reactive-codestart/codestart.yml

For the code of MyReactiveMessagingApplication, you have currently 2 channels source and uppercase both having inbound and outbound ends connected to each other.
That would work in-memory (without any connectors) but if we want to connect them to messaging backends via connectors we'll need to change the names of some of the channels (Two channels can't have the same name).

For example:
@Channel("source-out") Emitter -> outgoing connector
@Incoming("source") <- incoming connector
@Outgoing("uppercase-out") -> outgoing connector
@Incoming("uppercase") <- incoming connector

That would mean configuring 4 connectors inside application.properties. (In the near future if only one connector is available we'll configure them automatically)

Also, we can name the class according to the connector {Kafka, AMQP, RabbitMQ, etc}ReactiveMessagingApplication.

vgallet · 2021-10-31T06:59:19Z

@ia3andy @ozangunalp

Please can you take a look at this draft and how I created a dynamic application.yml. I also put some TODO into the code to show what's missing or if it's necessary to have it.

ia3andy

You also need to add the QuarkusCodestartTest, it will also help you make sure it works fine.

...resources/codestarts/quarkus/extension-codestarts/reactive-messaging-codestart/codestart.yml

...ion-codestarts/reactive-messaging-codestart/base/src/main/resources/application.tpl.qute.yml

...resources/codestarts/quarkus/extension-codestarts/reactive-messaging-codestart/codestart.yml

vgallet · 2021-11-15T08:17:45Z

@ia3andy @ozangunalp Hi,

I fixed your comments and added an integration test. Nevertheless, I added three TODO as I'm stuck to make it work and make it right :

Thanks for you help

ia3andy · 2021-11-15T10:53:14Z

I will check it out to give it a try and see what's wrong.

ia3andy · 2021-11-15T15:45:04Z

Ok, I've checked with @mkouba, we can't have a counter in the loop. I think we should not comment (until we find a proper way), the multi-connectors case is an edge case.

I will have another look tomorrow morning with a bit more time, let's wait until then..

ia3andy · 2021-11-16T10:05:03Z

Hey @vgallet, I prepared a commit with the fixes to make it work:
ia3andy@d9b66a8

I had to improve a bit the test framework to avoid conflicts between the different tests..

Let me know if you want me to push it to your branch directly..

ia3andy · 2021-11-16T10:10:56Z

I had to re-commit due to a problem: ia3andy@d9b66a8

ia3andy

LGTM

quarkus-bot · 2021-11-16T10:59:02Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

✖ This workflow run has failed but no jobs reported an error. Something weird happened, please check the workflow run page carefully: it might be an issue with the workflow configuration itself.

ozangunalp · 2021-11-16T14:24:18Z

@ia3andy would you know how to test codestarts in local?

ia3andy · 2021-11-16T14:32:49Z

@ozangunalp, you can either use the tests, or the CLI:
https://quarkus.io/version/main/guides/extension-codestart#generating

quarkus-bot · 2021-11-17T08:19:59Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `d83c72f`

Status	Name	Step	Failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	Failures	Logs	Raw logs

Failures

⚙️ Initial JDK 11 Build #

- Failing: integration-tests/devtools

📦 integration-tests/devtools

✖ Failed to execute goal net.revelc.code.formatter:formatter-maven-plugin:2.17.0:validate (default) on project quarkus-integration-test-devtools: File '/home/runner/work/quarkus/quarkus/integration-tests/devtools/src/test/java/io/quarkus/devtools/codestarts/quarkus/ReactiveMessagingCodestartIT.java' has not been previously formatted. Please format file and commit before running validation!

ia3andy · 2021-11-17T08:21:49Z

@vgallet could you run mvn process-sources and push?

* fix quarkusio#20872 * created `MyReactiveMessagingApplication` to demonstrates `@Emitter`, `@Incoming` and `@Outgoing` annotations. * created dynamic application.yml based on selected extension * added codestart integration test

quarkus-bot · 2021-11-17T10:48:33Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `c64c335`

Status	Name	Step	Failures	Logs	Raw logs
✖	Devtools Tests - JDK 11	`Build`	Failures	Logs	Raw logs
✖	Devtools Tests - JDK 11 Windows	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Devtools Tests - JDK 11 #

- Failing: integration-tests/devtools

📦 integration-tests/devtools

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testGradle(String)[1] line 78 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testRunTogetherCodestartsJava line 61 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testGradleKotlinDSL(String)[1] line 85 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testGradle(String)[1] line 78 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testRunTogetherCodestartsJava line 61 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testGradleKotlinDSL(String)[1] line 85 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

⚙️ Devtools Tests - JDK 11 Windows #

- Failing: integration-tests/devtools

📦 integration-tests/devtools

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testGradle(String)[1] line 78 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testRunTogetherCodestartsJava line 61 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testGradleKotlinDSL(String)[1] line 85 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildAll line 50 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testGradle(String)[1] line 78 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testRunTogetherCodestartsJava line 61 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.QuarkusCodestartBuildIT.testGradleKotlinDSL(String)[1] line 85 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 

expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildAll line 50 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

📦 integration-tests/devtools/target/quarkus-codestart-test/1637140781243/real-data/java

✖ ilove.quark.us.MyReactiveMessagingApplicationTest.test - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.kafka.client.deployment.DevServicesKafkaProcessor#startKafkaDevService threw an exception: java.lang.RuntimeException: java.lang.IllegalStateException: Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration

vgallet · 2021-11-25T13:55:54Z

Hello guys @ia3andy @ozangunalp! I discussed with @loicmathieu about the failing tests and he suggested me to split the test ReactiveMessagingCodestartIT into three tests, each dedicated per extension. WDYT ?

BTW, it misses the annotation @EnabledIfSystemProperty(named = "build-projects", matches = "true") on the test ReactiveMessagingCodestartIT#buildAll ?

ia3andy · 2021-11-25T15:21:47Z

Hello guys @ia3andy @ozangunalp! I discussed with @loicmathieu about the failing tests and he suggested me to split the test ReactiveMessagingCodestartIT into three tests, each dedicated per extension. WDYT ?

I will try to run the tests locally to see what's going on. I don't think we need to split.

BTW, it misses the annotation @EnabledIfSystemProperty(named = "build-projects", matches = "true") on the test ReactiveMessagingCodestartIT#buildAll ?

You don't need it, it your case you want that test to be run.

ia3andy · 2021-11-25T16:14:50Z

I pushed fixes, let see what the CI has to say to it :)

If it's green, could you squash it all? Thanks @vgallet ✌️

ia3andy · 2021-11-25T17:20:38Z

Ok the tests are failing because there is no docker environment on CI for the devservices to be started :-/ I am not sure what's the way forward..

ia3andy · 2021-11-25T17:26:20Z

It's weird that the same test is only failing on windows:

Error: ]: Build step io.quarkus.kafka.client.deployment.DevServicesKafkaProcessor#startKafkaDevService threw an exception: java.lang.RuntimeException: java.lang.IllegalStateException: Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration
	at io.quarkus.kafka.client.deployment.DevServicesKafkaProcessor.startKafkaDevService(DevServicesKafkaProcessor.java:116)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:887)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
	at java.base/java.lang.Thread.run(Thread.java:829)
	at org.jboss.threads.JBossThread.run(JBossThread.java:501)
Caused by: java.lang.IllegalStateException: Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration
	at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:114)

quarkus-bot · 2021-11-25T18:40:28Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `4bae693`

Status	Name	Step	Failures	Logs	Raw logs
✖	Devtools Tests - JDK 11 Windows	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Devtools Tests - JDK 11 Windows #

- Failing: integration-tests/devtools

📦 integration-tests/devtools

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildAll line 50 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildAll line 50 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

📦 integration-tests/devtools/target/quarkus-codestart-test/quarkus-smallrye-reactive-messaging-kafka-cb1a4d5c-075b-450f-a3c5-1eb4ea471a6a/real-data/java

✖ ilove.quark.us.MyReactiveMessagingApplicationTest.test - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.kafka.client.deployment.DevServicesKafkaProcessor#startKafkaDevService threw an exception: java.lang.RuntimeException: java.lang.IllegalStateException: Previous attempts to find a Docker environment failed. Will not retry. Please see logs and check configuration

ia3andy · 2021-11-26T13:06:14Z

@cescoffier do you know why this test is not working on Windows?

cescoffier · 2021-11-26T13:09:22Z

Nope, it should not spend on the OS

loicmathieu · 2021-11-26T13:58:40Z

@ia3andy Windows CI didn't have Docker, all IT that needs Docker are not run inside the Windows job.

ia3andy · 2021-11-26T17:51:30Z

Should we disable this build test on Windows then?
Or maybe we should disable the MyReactiveMessagingApplicationTest on Windows?

loicmathieu · 2021-11-27T18:04:20Z

As far as I know, all test that requires Docker are disabled in the CI.

ia3andy · 2021-12-01T07:22:37Z

Ok what do we suggest users in that matter? Maybe we should disable this test directly in the template so that users know it doesn't work on Windows? The problem is not related to the codestart test but more the generated project IMO.

ia3andy · 2021-12-01T08:06:51Z

@cescoffier what is our best practice for users regarding dev services using docker and Windows?

loicmathieu · 2021-12-01T08:21:16Z

@ia3andy Devservices works on Windows, I use them everyday.

But our Windows CI runner didn't have Docker so you must disabled the test via @DisableOnOs(OS.WINDOWS)

ia3andy · 2021-12-01T08:26:24Z

@DisableOnOs(OS.WINDOWS)

Thanks, I'll add this then :)

quarkus-bot · 2021-12-01T10:51:50Z

Failing Jobs - Building `eca87b6`

Status	Name	Step	Failures	Logs	Raw logs
✖	Devtools Tests - JDK 11	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Devtools Tests - JDK 11 #

- Failing: integration-tests/devtools

📦 integration-tests/devtools

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildAMQP line 65 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildMQTT line 59 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildKafka line 53 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildAMQP line 65 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildMQTT line 59 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

✖ io.quarkus.devtools.codestarts.quarkus.ReactiveMessagingCodestartIT.buildKafka line 53 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: 
[Run project return status is zero] 
expected: 0

ia3andy · 2021-12-01T13:49:06Z

@famod any idea why we don't run devtools tests also on JDK 17

ia3andy · 2021-12-02T18:00:03Z

I've found the problem and fixed it on my branch, @vgallet could you rebase from main so I can apply it?

famod · 2021-12-02T22:21:28Z

@ia3andy

@famod any idea why we don't run devtools tests also on JDK 17

Well, there simply wasn't a Java 16 job when I updated to 17. See also: #21481 (comment)

CI has so many jobs already, but as devtools ITs "only" take ~30m I think this could be added.

cc @gsmet

ia3andy · 2021-12-03T09:22:43Z

I am closing this one to be able to rebase and test with the CI (I am keeping you @vgallet as author): #21906

quarkus-bot bot added the area/devtools Issues/PR related to maven, gradle, platform and cli tooling/plugins label Oct 28, 2021

vgallet force-pushed the issue20872 branch from 2f16ca6 to aa29e9c Compare October 28, 2021 13:51

vgallet changed the title ~~Reactive Messaging codestart #20872~~ Reactive Messaging codestart Oct 28, 2021

vgallet force-pushed the issue20872 branch from aa29e9c to c1b9bca Compare October 28, 2021 13:52

vgallet marked this pull request as draft October 28, 2021 15:00

ozangunalp self-requested a review October 28, 2021 15:19

vgallet force-pushed the issue20872 branch from c1b9bca to 0d72a2e Compare October 31, 2021 06:55

ia3andy requested changes Nov 2, 2021

View reviewed changes

...resources/codestarts/quarkus/extension-codestarts/reactive-messaging-codestart/codestart.yml Outdated Show resolved Hide resolved

ozangunalp requested changes Nov 3, 2021

View reviewed changes

vgallet force-pushed the issue20872 branch from 0d72a2e to e97bdb8 Compare November 15, 2021 08:09

quarkus-bot bot added area/reactive-messaging area/smallrye labels Nov 15, 2021

quarkus-bot bot added the area/platform Issues related to definition and interaction with Quarkus Platform label Nov 16, 2021

vgallet force-pushed the issue20872 branch from aef52ff to d83c72f Compare November 16, 2021 10:33

vgallet marked this pull request as ready for review November 16, 2021 10:42

ia3andy approved these changes Nov 16, 2021

View reviewed changes

Reactive Messaging codestart

c64c335

* fix quarkusio#20872 * created `MyReactiveMessagingApplication` to demonstrates `@Emitter`, `@Incoming` and `@Outgoing` annotations. * created dynamic application.yml based on selected extension * added codestart integration test

vgallet force-pushed the issue20872 branch from d83c72f to c64c335 Compare November 17, 2021 08:33

Improve testId in QuarkusCodestartTest and fix QuarkusCodestartBuildIT

4bae693

Disable build test on Windows because no docker on CI

eca87b6

famod mentioned this pull request Dec 2, 2021

Run devtools ITs also with Java 17 #21903

Merged

ia3andy closed this Dec 3, 2021

quarkus-bot bot added the triage/invalid This doesn't seem right label Dec 3, 2021

Reactive Messaging codestart #21068

Reactive Messaging codestart #21068

Conversation

vgallet commented Oct 28, 2021 • edited Loading

quarkus-bot bot commented Oct 28, 2021

vgallet commented Oct 28, 2021

ia3andy commented Oct 28, 2021

vgallet commented Oct 28, 2021 • edited Loading

ozangunalp commented Oct 28, 2021

vgallet commented Oct 31, 2021

ia3andy left a comment

Choose a reason for hiding this comment

vgallet commented Nov 15, 2021

ia3andy commented Nov 15, 2021

ia3andy commented Nov 15, 2021 • edited Loading

ia3andy commented Nov 16, 2021 • edited Loading

ia3andy commented Nov 16, 2021 • edited Loading

ia3andy left a comment

Choose a reason for hiding this comment

quarkus-bot bot commented Nov 16, 2021 • edited Loading

ozangunalp commented Nov 16, 2021

ia3andy commented Nov 16, 2021

quarkus-bot bot commented Nov 17, 2021 • edited Loading

Failing Jobs - Building d83c72f

Failures

⚙️ Initial JDK 11 Build #

📦 integration-tests/devtools

ia3andy commented Nov 17, 2021

quarkus-bot bot commented Nov 17, 2021 • edited Loading

Failing Jobs - Building c64c335

Failures

⚙️ Devtools Tests - JDK 11 #

📦 integration-tests/devtools

⚙️ Devtools Tests - JDK 11 Windows #

📦 integration-tests/devtools

📦 integration-tests/devtools/target/quarkus-codestart-test/1637140781243/real-data/java

vgallet commented Nov 25, 2021

ia3andy commented Nov 25, 2021 • edited Loading

ia3andy commented Nov 25, 2021 • edited Loading

ia3andy commented Nov 25, 2021

ia3andy commented Nov 25, 2021

quarkus-bot bot commented Nov 25, 2021 • edited Loading

Failing Jobs - Building 4bae693

Failures

⚙️ Devtools Tests - JDK 11 Windows #

📦 integration-tests/devtools

📦 integration-tests/devtools/target/quarkus-codestart-test/quarkus-smallrye-reactive-messaging-kafka-cb1a4d5c-075b-450f-a3c5-1eb4ea471a6a/real-data/java

ia3andy commented Nov 26, 2021

cescoffier commented Nov 26, 2021

loicmathieu commented Nov 26, 2021

ia3andy commented Nov 26, 2021

loicmathieu commented Nov 27, 2021

ia3andy commented Dec 1, 2021

ia3andy commented Dec 1, 2021

loicmathieu commented Dec 1, 2021

ia3andy commented Dec 1, 2021

quarkus-bot bot commented Dec 1, 2021

Failing Jobs - Building eca87b6

Failures

⚙️ Devtools Tests - JDK 11 #

📦 integration-tests/devtools

ia3andy commented Dec 1, 2021

ia3andy commented Dec 2, 2021

famod commented Dec 2, 2021 • edited Loading

ia3andy commented Dec 3, 2021

vgallet commented Oct 28, 2021 •

edited

Loading

vgallet commented Oct 28, 2021 •

edited

Loading

ia3andy commented Nov 15, 2021 •

edited

Loading

ia3andy commented Nov 16, 2021 •

edited

Loading

ia3andy commented Nov 16, 2021 •

edited

Loading

quarkus-bot bot commented Nov 16, 2021 •

edited

Loading

quarkus-bot bot commented Nov 17, 2021 •

edited

Loading

Failing Jobs - Building `d83c72f`

quarkus-bot bot commented Nov 17, 2021 •

edited

Loading

Failing Jobs - Building `c64c335`

ia3andy commented Nov 25, 2021 •

edited

Loading

ia3andy commented Nov 25, 2021 •

edited

Loading

quarkus-bot bot commented Nov 25, 2021 •

edited

Loading

Failing Jobs - Building `4bae693`

Failing Jobs - Building `eca87b6`

famod commented Dec 2, 2021 •

edited

Loading