clusterctl create sometimes fails to create cluster object

[krousey]

Usually appears like:

F0622 10:16:44.593398  128171 create_cluster.go:58] unable to create cluster object: Timeout: request did not complete within allowed duration

[krousey]

/assign @krousey

[roberthbailey]

This has been happening to me quite frequently this week.

Environment:

macOS 10.13.5
minikube 0.28.0 (and then also 0.27.0)
VirtualBox Version 5.2.12 r122591 (Qt5.6.3)

If I give minikube more memory & cpu, then it seems more likely that the call doesn't time out, but it's still hit or miss.

[krousey]

This is happening, contrary to my earlier assumptions, because the cluster-api API server comes up and registers the cluster API group with the main server before it's backing ETCD store is ready. This is a race, but exacerbated by the fact that we pull from quay's latest tag, and always pull.

I can reliably reproduce this by changing the image field to something that can't be pulled.

[krousey]

I'm going to change the image, and add some more detection for readiness so we can avoid this.

[krousey]

Also a recent switch from Poll to PollImmediate made the race more likely. This is probably the reason some of us were seeing this more frequently. I had been using an old version that didn't have this change.