Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve build performance of modules #12485

Closed
MC-JY opened this issue Apr 26, 2022 · 5 comments · Fixed by #12486
Closed

Improve build performance of modules #12485

MC-JY opened this issue Apr 26, 2022 · 5 comments · Fixed by #12486

Comments

@MC-JY
Copy link
Contributor

MC-JY commented Apr 26, 2022

Hi, I'm now using druid. I found that the build time of the project is not very fast when I used mvn -T 1C install -DskipTests command to build the project during Github Actions. So I try to speed up the build performance of the project. The goal here is to clean up dependencies between maven modules in the project in order to improve the build performance.

Thread usage before cleaning up module dependencies was this:
1

Using mvn dependency:analyze command I managed to have an overview of the unused dependencies of every module. I focused only on the dependencies between modules. The following dependencies can be cleaned up:


org.apache.druid.extensions:druid-basic-security(druid/extensions-core/druid-basic-security) -> org.apache.druid:druid-services(druid/services)
org.apache.druid:druid-server(druid/server) -> org.apache.druid:druid-console(druid/web-console)

After clean up, the build time has been reduced by 3min7s when I use parallel build during Github Actions.
Thread usage after cleaning up module dependencies was this:
2

Could you help me review this issue? I can submit a PR to improve build performance of the project.

Thank you very much for your attention.
Best regards.

@2bethere
Copy link
Contributor

What's the before time? And yes please if you can submit a PR it'll be great!

@MC-JY
Copy link
Contributor Author

MC-JY commented Apr 26, 2022

The time before modification is 10 minutes when I used mvn -T 1C install -DskipTests command to build the project during Github Actions. Sure, I can submit a PR to improve build performance.

@FrankChen021
Copy link
Member

A PR is welcomed.

And what's the rationale that speeds up the build time? I'm curious that which tool are you using to get the thread usage during building.

@FrankChen021
Copy link
Member

I'm wondering if we can make some chanes to current dependency check so that these dependency problems can be exposed earlier at least at the CI phase.

Druid's root pom contains dependency plugin declaration, but only these modules configure to use dependency check
image

So, it's better to enable the dependency check at the root pom level so that we don't need to configure it for each module.

@MC-JY
Copy link
Contributor Author

MC-JY commented Apr 28, 2022

Yes, it's better to enable the dependency check at the root pom level. Only required modules will use the plugin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants