Non-nil default AzureMachine spec.diagnostics breaks v1alpha4 -> v1beta1 upgrade test

[nojnhuh]

/kind bug
/kind failing-test

What steps did you take and what happened:

The capz-periodic-apiversion-upgrade-main test has been failing consistently the past couple of days (example), timing out waiting for the cluster to be deleted. The capz-controller-manager logs show a few instances of this error:

E1215 21:11:11.953273       1 controller.go:326]  "msg"="Reconciler error" "error"="admission webhook \"validation.azuremachine.infrastructure.cluster.x-k8s.io\" denied the request: AzureMachine.infrastructure.cluster.x-k8s.io \"clusterctl-upgrade-wdwduc-md-0-pf47n\" is invalid: Spec.Diagnostics: Invalid value: v1beta1.Diagnostics{Boot:(*v1beta1.BootDiagnostics)(0xc000de1668)}: field is immutable, unable to assign a value if it was already empty" "AzureMachine"={"name":"clusterctl-upgrade-wdwduc-md-0-pf47n","namespace":"clusterctl-upgrade"} "controller"="azuremachine" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureMachine" "name"="clusterctl-upgrade-wdwduc-md-0-pf47n" "namespace"="clusterctl-upgrade" "reconcileID"="ade2fce0-5f16-44c5-9aca-cbccc7cba325"

What did you expect to happen:

Anything else you would like to add:

It looks like the changes in #2528 are responsible for the new failures. I think the issue is that a v1alpha4 AzureMachine will be created with an empty spec.diagnostics since that field is doesn't exist there, but the default value added by the v1beta1 webhook is not empty and the validating webhook enforces that spec.diagnostics is immutable.

I was able to reproduce the error locally and after removing the defaulting here, the test passes:

cluster-api-provider-azure/api/v1beta1/azuremachine_default.go

Line 127 in c1b8d85

s.SetDiagnosticsDefaults()

Environment:

• cluster-api-provider-azure version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[jackfrancis]

/assign

[CecileRobertMichon]

/assign @nawazkh
/unassign @jackfrancis