e2e test for k8s v1.22 --> v1.23 upgrade to use out of tree csi driver

[sonasingh46]

/kind bug

What steps did you take and what happened:
From Kubernetes v1.23 AzureDiskCSI driver is enabled by default.

An e2e test with following validations should be there:

• Create a k8s cluster with v1.22.
• Create a deployment/sts that uses PVC created via in-tree AzureDiskCSI driver.
• Upgrade the k8s cluster to v1.23 and install out-of-tree AzureDiskCSI driver.
• Validate that the existing pod using PVC created via in-tree provider keeps functioning.
• Validate that a new pod using PVC created via the out-of-tree provider works.

What did you expect to happen:

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• cluster-api-provider-azure version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[sonasingh46]

/assign sonasingh46

[sonasingh46]

cc @CecileRobertMichon @shysank

[CecileRobertMichon]

@sonasingh46 thanks for picking this up.

Can we leverage Helm to install AzureDisk and AzureFile on the cluster similar to what we did in #2209 to install external cloud provider?

https://github.com/kubernetes-sigs/azurefile-csi-driver/tree/master/charts#install-csi-driver-with-helm-3
https://github.com/kubernetes-sigs/azuredisk-csi-driver/tree/master/charts#install-csi-driver-with-helm-3

We should also add user-facing docs about installing CSI drivers post cluster create.

[sonasingh46]

@CecileRobertMichon -- Yes. I am planning to use helm like we did for external cloud provider.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[jackfrancis]

/remove-lifecycle stale