Avoid expesive pointer copy in capi nodegroup

[enxebre]

What type of PR is this?

/kind bug
/kind cleanup

What this PR does / why we need it:

Autoscaler only chooses and scales up a nodegroup each run. Meaning that if you have multiple unschedulable pods targeting different groups each of them (e.g. via nodeSelector), autoscaler would only be able to find a place for them sequentially, therefore having a penalty time (that keeps degrading as the number of pods/nodegroups increases) for the latest pods in "the queue". When having ~90 scalable resources with capi provider this results in long times to provision nodes for latests pods in the "queue" (>40min).

The func (p *provider) NodeGroups() []cloudprovider.NodeGroup { is called around 16 times every runOnce loop. Currently it takes ~20 seconds with ~90 MachineSets. This avoids expensive loop and copy pointers resulting in ~5 seconds each NodeGroups call, and so in ~4min saving each run.

Which issue(s) this PR fixes:

Mitigates #6784

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[enxebre]

/hold
a bit

[elmiko]

nice optimization @enxebre , i hesitated a little thinking about the log line but reviewing my old log files i think that line is not as useful as we might have thought originally.

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: elmiko, enxebre

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster-autoscaler/cloudprovider/clusterapi/OWNERS [elmiko,enxebre]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[elmiko]

/lgtm

[enxebre]

some benchMarks
before

➜  cluster-autoscaler git:(release-4.16) ✗ go test -v -bench=. -count 5 -run=NodeGroupsBenchmark ./cloudprovider/clusterapi/...
goos: darwin
goarch: amd64
pkg: k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkNodeGroups
I0509 16:35:53.758484   50802 clusterapi_controller.go:426] Using version "v1alpha3" for API group "cluster.x-k8s.io"
I0509 16:35:53.758676   50802 clusterapi_controller.go:541] Resource "machinedeployments" available
I0509 16:35:53.758740   50802 clusterapi_controller.go:541] Resource "machinedeployments" available
I0509 16:35:53.758749   50802 clusterapi_controller.go:541] Resource "machinesets" available
I0509 16:35:53.758754   50802 clusterapi_controller.go:541] Resource "machines" available
I0509 16:35:53.758760   50802 clusterapi_controller.go:541] Resource "machinepools" available
BenchmarkNodeGroups/NodeGroups
BenchmarkNodeGroups/NodeGroups-12         	     798	   1417817 ns/op
BenchmarkNodeGroups/NodeGroups-12         	     814	   1447729 ns/op
BenchmarkNodeGroups/NodeGroups-12         	     711	   1454166 ns/op
BenchmarkNodeGroups/NodeGroups-12         	     777	   1422128 ns/op
BenchmarkNodeGroups/NodeGroups-12         	     788	   1486356 ns/op
PASS
ok  	k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi	7.960s

After

➜  cluster-autoscaler git:(capi-nodegroups) ✗ go test -v -bench=. -count 5 -run=NodeGroupsBenchmark ./cloudprovider/clusterapi/...
goos: darwin
goarch: amd64
pkg: k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkNodeGroups
I0509 16:32:44.577941   50440 clusterapi_controller.go:426] Using version "v1alpha3" for API group "cluster.x-k8s.io"
I0509 16:32:44.578106   50440 clusterapi_controller.go:543] Resource "machinedeployments" available
I0509 16:32:44.578137   50440 clusterapi_controller.go:543] Resource "machinedeployments" available
I0509 16:32:44.578142   50440 clusterapi_controller.go:543] Resource "machinesets" available
I0509 16:32:44.578145   50440 clusterapi_controller.go:543] Resource "machines" available
I0509 16:32:44.578147   50440 clusterapi_controller.go:543] Resource "machinepools" available
BenchmarkNodeGroups/NodeGroups
BenchmarkNodeGroups/NodeGroups-12         	    1215	    948669 ns/op
BenchmarkNodeGroups/NodeGroups-12         	    1120	   1011799 ns/op
BenchmarkNodeGroups/NodeGroups-12         	    1159	   1052832 ns/op
BenchmarkNodeGroups/NodeGroups-12         	     999	   1023880 ns/op
BenchmarkNodeGroups/NodeGroups-12         	    1122	   1020194 ns/op
PASS
ok  	k8s.io/autoscaler/cluster-autoscaler/cloudprovider/clusterapi	7.701s

[enxebre]

/hold cancel

[elmiko]

very cool addition of the benchmarks, thanks!

/lgtm