Add E2E test to verify workload cluster upgrades

[srm09]

User Story

As a developer, I would like to run an E2E test to verify upgrading CAPI workload cluster between 2 supported versions.

Detailed Description

• A new prow job should be run periodically which runs the entire upgrade matrix for workload clusters.
• As for the actual test, we carry out the following steps
  • Create a workload cluster with version, say v.1.17
  • Upgrade the workload cluster to version, say v1.18
  • Run conformance tests on the upgraded cluster to verify the normal working of the cluster post upgrade.
  • As @fabriziopandini commented below, upgrade refers to upgrading the CP, MachinePool and MachineDeployment worker nodes.

Anything else you would like to add:

For the complete discussion on the introduction of these tests, look at #3693 (comment) .
TL;DR We need to add E2E tests to verify the upgrades between supported k8s versions.

/kind feature

[srm09]

/assign
Waiting for the PRs #3916 and #4042 to get merged.
cc: @fabriziopandini

[vincepri]

Any reason we've picked these two specific Kubernetes versions?

[vincepri]

/milestone v0.4.0

[detiber]

Any reason we've picked these two specific Kubernetes versions?

+1, While I think it makes sense to have something as the defaults, I would like to see something that is more easily configured at runtime that would allow for automating prow jobs more easily.

[srm09]

The idea was to start with a single job which tests one of the upgrade paths mentioned in the #3693 (comment). Once that gets done, we raise new issues for all the other upgrade tests required and ask some help from the community to finish those off.

I would like to see something that is more easily configured at runtime that would allow for automating prow jobs more easily.

That is the end goal, to keep the test/job configurable enough to introduce more upgrade paths easily.

[fabriziopandini]

As a reminder, upgrade in this context should include upgrading the control plane and the workers (including machines controlled by MachineDeployments and machines controlled by MachinePools)

[fabriziopandini]

#3916 merged, so now this activity is unblocked
/help

[k8s-ci-robot]

@fabriziopandini:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

#3916 merged, so now this activity is unblocked
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[srm09]

Updating this story to include the entire test matrix for workload cluster upgrade testing.

[srm09]

/retitle Add E2E test to verify workload cluster upgrades

[fabriziopandini]

/lifecycle active
/priority important-soon

[vincepri]

Is this being worked on?

[fabriziopandini]

yes
There are PR both in CAPI and test infra ready to merge (I pinged recently in slack you and @CecileRobertMichon)