[release-1.4] 🐛 requeue KCP object if ControlPlaneComponentsHealthyCondition is not yet true

[k8s-infra-cherrypick-robot]

This is an automated cherry-pick of #9032

/assign sbueringer

[killianmuldoon]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: killianmuldoon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [killianmuldoon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 20d1e24e82b4c020a135f45e7978b3fb70e1775a

[chrischdi]

/hold

as the issue comes only with PreFlight checks which are there beginning with v1.5 🤔

[chrischdi]

Also: only visible on v1.5 and main:

https://storage.googleapis.com/k8s-triage/index.html?text=Timed%20out%20waiting%20for%201%20nodes%20to%20be%20created%20for%20MachineDeployment&job=.*-cluster-api-.*&xjob=.*-provider-.*

[chrischdi]

/retest

[killianmuldoon]

I think the assessment was that this issue does happen on 1.3 and 1.4 - but it's surfaced by the preflight checks in our e2e tests. With preflight checks off this issue still occurs and reconciliation is still slower than it should be, but it doesn't block the reconciliation of MachineSets.

[chrischdi]

/hold cancel

[sbueringer]

/approve

[killianmuldoon]

/area provider/control-plane-kubeadm