Optimize CAPD machine controller to reduce noisy error messages

[sbueringer]

When implementing #7963 we discoverd that the CAPD machine reconciler is producing a lot of error messages when the APIserver of the workload cluster is not reachable yet. Roughly here: https://github.com/kubernetes-sigs/cluster-api/blob/main/test/infrastructure/docker/internal/controllers/dockermachine_controller.go#L337-L340

We tried to add the following statement above, but this breaks the standalone machine case (covered by the KCP adoption e2e test):

	// If the control plane is not yet initialized, there is no API server to contact to get the ProviderID for the Node
	// hosted on this machine, so return early.
	// NOTE: we are using RequeueAfter with a short interval in order to make test execution time more stable.
	if !conditions.IsTrue(cluster, clusterv1.ControlPlaneInitializedCondition) {
		return ctrl.Result{RequeueAfter: 15 * time.Second}, nil
	}

The problem is that in the standalone case we have to set the providerID and only afterwards the ControlPlaneInitialized condition is set to true. So with this statement we're in a deadlock.

Options:

• only requeue if the cluster is using a control plane provider

/kind cleanup

[sbueringer]

cc @fabriziopandini
Hope I captured it correctly

[fabriziopandini]

/triage accepted

only requeue if the cluster is using a control plane provider

Makes sense for me

[sbueringer]

/assign

Should have a few minutes to open a PR relatively soon