Refactor scale-up to apply resource limits before creating a node group

[aleksandra-malinowska]

Small refactor prior to the implementation of atomic scale-up support for Provisioning Requests: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/proposals/provisioning-request.md

What this PR does / why we need it:

Apply resource limits to scale-up plan before creating a new node group. Note that this is independent of the steps that happen immediately before and after - newNodes aren't used until we balance similar node groups, and non-existent node groups already have to expose node resources (otherwise scale-up simulations wouldn't be possible).

There's already a check in place for skipping a node group if adding even one node would exceed resource limits, so it's not technically a bug right now, and so this PR doesn't change current behavior. However, for the atomic scale up support for Provisioning Requests, we'll want to apply resource limits before taking any action, including creating a new node group.

Does this PR introduce a user-facing change?

NONE

[aleksandra-malinowska]

/cc @yaroslava-serdiuk @towca

[yaroslava-serdiuk]

/lgtm

[towca]

/lgtm
/approve
/hold

nit: I'd consider adding a test for this behavior (limits exceeded -> ScaleUp error without attempting to create a node group). Feel free to unhold if you don't think it's important

[aleksandra-malinowska]

nit: I'd consider adding a test for this behavior (limits exceeded -> ScaleUp error without attempting to create a node group). Feel free to unhold if you don't think it's important

Added test. Note: while OnGroupCreate isn't used in this test case, I needed to add it to confirm the test indeed works (would trigger autoprovisioning if max cores limit was increased) and decided to left it as I'd have to add it another PR soon anyway.

[towca]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aleksandra-malinowska, towca

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster-autoscaler/OWNERS [towca]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[aleksandra-malinowska]

@towca added the requested unit test, lost lgtm, can you PTAL? Thanks!

[towca]

It's the hold!
/unhold

[yaroslava-serdiuk]

#6815

[yaroslava-serdiuk]

/cherry-pick cluster-autoscaler-release-1.30

[k8s-infra-cherrypick-robot]

@yaroslava-serdiuk: new pull request created: #6931

In response to this:

/cherry-pick cluster-autoscaler-release-1.30

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[yaroslava-serdiuk]

/assign @towca