Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container image and Kubernetes ITs takes too much time on CI #31273

Closed
gsmet opened this issue Feb 18, 2023 · 9 comments
Closed

Container image and Kubernetes ITs takes too much time on CI #31273

gsmet opened this issue Feb 18, 2023 · 9 comments
Assignees
Labels
area/infra-automation anything related to CI, bots, etc. that are used to automated our infrastructure area/kubernetes

Comments

@gsmet
Copy link
Member

gsmet commented Feb 18, 2023

I recently had a look at how the time in CI is spread and stumbled upon this:

2023-02-17T21:32:22.0244219Z [INFO] Quarkus - Integration Tests - Container Image - Invoker SUCCESS [13:17 min]
2023-02-17T21:32:22.0244682Z [INFO] Quarkus - Integration Tests - Container Image - Standard SUCCESS [ 19.944 s]
2023-02-17T21:32:22.0245125Z [INFO] Quarkus - Integration Tests - Kubernetes - Invoker . SUCCESS [01:15 min]
2023-02-17T21:32:22.0245567Z [INFO] Quarkus - Integration Tests - Kubernetes - Standard  SUCCESS [09:14 min]

That is 20+ minutes to run the Container image and Kubernetes ITs.

If this is normal (?), we need to split them from the standard JDK builds as we did for the Maven and Gradle tests.

@gsmet gsmet changed the title Cl ta Container image and Kubernetes ITs taken too much time on CI Feb 18, 2023
@gsmet gsmet added area/kubernetes area/infra-automation anything related to CI, bots, etc. that are used to automated our infrastructure and removed triage/needs-triage labels Feb 18, 2023
@quarkus-bot
Copy link

quarkus-bot bot commented Feb 18, 2023

/cc @Sgitario (kubernetes), @geoand (knative,kubernetes,minikube,openshift), @iocanel (knative,kubernetes,openshift)

@gsmet
Copy link
Member Author

gsmet commented Feb 18, 2023

@iocanel could you check if anything suspicious is going on there?

I checked the raw logs of this build: https://github.com/quarkusio/quarkus/actions/runs/4206183467/jobs/7299751359

@gsmet gsmet changed the title Container image and Kubernetes ITs taken too much time on CI Container image and Kubernetes ITs takes too much time on CI Feb 18, 2023
@iocanel
Copy link
Contributor

iocanel commented Feb 20, 2023

Started looking at the issue.

@iocanel iocanel self-assigned this Feb 20, 2023
@iocanel
Copy link
Contributor

iocanel commented Feb 20, 2023

@gsmet: I looked in detail at the Kubernetes tests and I don't see anything worrisome other than their number.

The problem seems to lie in the container image tests (the invoker part which does take the longest).

I think that we do have way too many of them that practically test the same thing.
For example, I don't see why we test the jib build along with redis, keycloak, mongo, etc as practically the additional dependencies only affect the content of the jar to be containerized and nothing more.

So, my suggestion is to keep one for each extension (one docker, one jib etc) and drop the rest. If we want to furhter trim the time down we could just run them daily as we do with the kubernetes ones.

@geoand: wdyt?

@geoand
Copy link
Contributor

geoand commented Feb 20, 2023

Sounds reasonable to me

@Sgitario
Copy link
Contributor

@iocanel This issue should have not been addressed by #31295 ? Or are you planning to do something else?

@iocanel
Copy link
Contributor

iocanel commented Feb 21, 2023

Depends on whether we are satisfied with the result. If not we could run the container itests once per day. @gsmet how does it look now?

@iocanel
Copy link
Contributor

iocanel commented Mar 20, 2023

@gsmet: wdyt is this something we can close now?

@iocanel
Copy link
Contributor

iocanel commented Aug 4, 2023

Let's close this and we can re-open if issue re-appears

@iocanel iocanel closed this as completed Aug 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/infra-automation anything related to CI, bots, etc. that are used to automated our infrastructure area/kubernetes
Projects
None yet
Development

No branches or pull requests

4 participants