Allow running e2e tests as presubmits on GitHub

[kwiesmueller]

We recently had the case where a PR (#5911) needed to be rolled back because e2e tests were failing.
The author was not able to run the e2e tests and in general I don't think we should rely on people to run e2e tests locally before we accept the code.

I'd propose to move some of our E2E tests into presubmits (https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-autoscaling/sig-autoscaling-config.yaml) and at least allow to trigger them via bot commands.

This could also allow us to separate CI runs between cluster-autoscaler and VPA as the config allows us to specify which tests to run on which changes.

/good-first-issue
/help

[k8s-ci-robot]

@kwiesmueller:
This request has been marked as suitable for new contributors.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.

In response to this:

We recently had the case where a PR (#5911) needed to be rolled back because e2e tests were failing.
The author was not able to run the e2e tests and in general I don't think we should rely on people to run e2e tests locally before we accept the code.

I'd propose to move some of our E2E tests into presubmits (https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-autoscaling/sig-autoscaling-config.yaml) and at least allow to trigger them via bot commands.

This could also allow us to separate CI runs between cluster-autoscaler and VPA as the config allows us to specify which tests to run on which changes.

/good-first-issue
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[uroy-personal]

Hi Kevin,
I am interested to work on this issue. I am trying to retrieve as much details as possible from different links present in this issue. Would be highly obliged if you kindly share more info on this issue. Shall we discuss in slack quickly?

[jbartosik]

I like the idea

[kwiesmueller]

Feel free to reach out on Slack (kwiesmueller) for more details too.
I only have limited context on the infra side, but #sig-test-infra should be able to help with answers too.
Aside from that I think the config linked above is where the magic happens.

[uroy-personal]

/assign @uroy-personal

[uroy-personal]

I like the idea

I like the idea

Thanks @jbartosik !

[uroy-personal]

Thank you @kwiesmueller , I have observed the differences between these two formats. Hope I will be able to address this change now. As part of this issue how many test cases from https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-autoscaling/sig-autoscaling-config.yaml (it has around 10 test cases ) should I move to https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-autoscaling/sig-autoscaling-presubmits.yaml ( 3 test cases )? . Or shall I just perform a proof of concept with just one test case first. Please advise. Thanks again!

[kwiesmueller]

I think a PoC with one test each (one for VPA, one for Cluster Autoscaler) makes sense before you do all the work for the others. Maybe we can get these submitted fast to try them on real PRs before moving forward with the rest.

[kwiesmueller]

For VPA I'd say let's start with ci-kubernetes-e2e-autoscaling-vpa-full as it's the most comprehensive.
I can't see a test for CA, maybe @mwielgus has a suggestion for a CA test to run on presubmit, otherwise we can only try VPA for now.

[uroy-personal]

Much appreciated @kwiesmueller ,
I hope to complete ci-kubernetes-e2e-autoscaling-vpa-full in a day or two, raise the PR and liaise with you again. Thanks!

[uroy-personal]

Hi @kwiesmueller ,
I have raised the PR with changes for the ci-kubernetes-e2e-autoscaling-vpa-full. Need some assistance in the review of this PR : kubernetes/test-infra#30891 . I am following up with the reviewer group.