Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the number of vert.x eventloops started by default #27025

Merged
merged 3 commits into from
Aug 19, 2022

Conversation

Sanne
Copy link
Member

@Sanne Sanne commented Jul 29, 2022

As discussed at JNation, we have some experiments showing that:

  • there's no compelling reason to start more eventloops than available cores
  • it's actually harmful for performance in some cases

And obviously it saves a little chunk of memory as well. We briefly discussed the possibility to create "named profiles" so that people could experiment with different scaling formulas, but we decided to keep it simple: this is just the default meant to provide a reasonable OOB experience, while people wanting to tune things further can set the appropriate configuration property.

cc/ @franz1981

Copy link
Contributor

@franz1981 franz1981 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, as said before, this is going to save some memory resources eg TLAB of Netty pooled allocators, EPOLL selectors and native threads
Additionally it's going to return back resources for IRQ processing and JIT background work

@quarkus-bot

This comment has been minimized.

@technical-debt-collector
Copy link
Contributor

technical-debt-collector commented Jul 30, 2022

This is really interesting.
@Sanne, are the experiments available somewhere? I found some JNation videos but couldn't find anything on this.

Imo this is an important topic with generally too little information available - there's hardly anything beyond the description of the config parameter. If there's reason to either adjust the configuration or leave it alone, then I think developers should know about it.
In our case we now have two reactive applications, one of which cannot be run with multiple instances - and we've upped the event loop pool size for the latter :)

Vert.x still runs with the 2 x core-default. Do the guys over there know about these experiments?
https://github.com/eclipse-vertx/vert.x/blob/428f2fd0966057eb9f0e83f70ada8dc08c092a0a/src/main/java/io/vertx/core/VertxOptions.java

@Sanne
Copy link
Member Author

Sanne commented Jul 31, 2022

Hi @technical-debt-collector - I'm referring to our experiments with the Techempower benchmarks; you can easily run them yourself, would be great to have some more eyes on these:

Specifically the code we were looking into is in:

We discuss some of these things in a recent Quarkus Insights as well:

Yes I know about the vert.x defaults - we're talking and working with the vert.x team as well, however since that's more exposed in that case one might not want to make such a change in a minor release; in Quarkus however we think it's safe enough.

BTW I'll be out of town for some days - might be slow to follow up :) @franz1981 knows more about this though, feel free to discuss here.

@geoand
Copy link
Contributor

geoand commented Aug 1, 2022

The CI failures seem related

@technical-debt-collector
Copy link
Contributor

Thanks @Sanne! I'll have a look at this as soon as I find the time, possibly not before EOW.

@quarkus-bot

This comment has been minimized.

@quarkus-bot

This comment has been minimized.

@Sanne
Copy link
Member Author

Sanne commented Aug 9, 2022

I fixed the issue in the vertx tests, which were making assumptions about the number of threads we'd be starting.

But now there's some new tests failing, Windows only - I'll need to check those later. Does anyone have an effective way to check if these tests have been failing recently in other PRs ?

I'm puzzled about why this patch would break something only on Windows.

@gsmet
Copy link
Member

gsmet commented Aug 9, 2022

I don't remember seeing this one failing. One way to be sure is to just do a search in the PR with the name of the test, given they are included in comments. That's one nice additional benefit of the reports.

And apparently, it also failed here: #27210 .

@Sanne
Copy link
Member Author

Sanne commented Aug 10, 2022

so you're searching like is:pr is:open "io.quarkus.it.opentelemetry.vertx.HelloRouterTest.bus" ?

I couldn't find any previous similar failure; I assume you're referring to the other failure? in that case it's fairly ancient, not sure if it should count.

Sanne added 3 commits August 10, 2022 21:48

Verified

This commit was signed with the committer’s verified signature.
gsmet Guillaume Smet
@Sanne
Copy link
Member Author

Sanne commented Aug 10, 2022

I've finally been able to run tests on a local Windows box - of course they don't fail here :(

@gsmet
Copy link
Member

gsmet commented Aug 10, 2022

I just search for HelloRouterTest in the search box for instance.

@quarkus-bot
Copy link

quarkus-bot bot commented Aug 11, 2022

Failing Jobs - Building 9bf91d3

Status Name Step Failures Logs Raw logs
Gradle Tests - JDK 11 Build Failures Logs Raw logs
✔️ Gradle Tests - JDK 11 Windows
JVM Tests - JDK 11 Build Failures Logs Raw logs
✔️ JVM Tests - JDK 17
✔️ JVM Tests - JDK 18

Full information is available in the Build summary check run.

Failures

⚙️ Gradle Tests - JDK 11 #

- Failing: integration-tests/gradle 

📦 integration-tests/gradle

io.quarkus.gradle.devmode.CompositeBuildWithDependenciesDevModeTest.main line 24 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference, java.util.concurrent.atomic.AtomicReferencejava.lang.String, java.lang.Stringboolean was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

⚙️ JVM Tests - JDK 11 #

- Failing: extensions/smallrye-reactive-messaging-amqp/deployment 
! Skipped: integration-tests/reactive-messaging-amqp 

📦 extensions/smallrye-reactive-messaging-amqp/deployment

io.quarkus.smallrye.reactivemessaging.amqp.SecuredAmqpTest.test line 28 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with io.quarkus.smallrye.reactivemessaging.amqp.SecuredAmqpTest was not fulfilled within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

@Sanne
Copy link
Member Author

Sanne commented Aug 11, 2022

The remaining two failures seem to be frequent offenders, so I think this is ready to be merged.

@Sanne Sanne merged commit 4d511de into quarkusio:main Aug 19, 2022
@quarkus-bot quarkus-bot bot added this to the 2.13 - main milestone Aug 19, 2022
@Sanne Sanne deleted the LowerIOThreads branch August 19, 2022 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants