Switch to Standard_B2s VM SKU by default

[mboersma]

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

An internal Azure audit suggested some of our accounts try to use Standard_B2s VM SKUs instead of our default Standard_D2s_v3. I've been using it locally for all my testing and development and haven't seen any problems, so let's see if it might work for others (in particular, our CI account).

Which issue(s) this PR fixes:

N/A

Special notes for your reviewer:

The difference in the two VM SKUs is this:

SKU	Cores	Memory	Storage	Price
B2s	2	4GB RAM	8GB temp	$0.058 / hr
D2s_v3	2	8GB RAM	16GB temp	$0.209 / hr

TODOs:

• squashed commits
• includes documentation
• adds unit tests

Release note:

Switch to Standard_B2s VM SKU by default

[mboersma]

/hold

Even if this PR passes, we need to check that the test account has sufficient quota and that there is availability in most regions.

[mboersma]

So close, but AKS doesn't like it:

E1027 19:56:50.700817       1 controller.go:326]  
"msg"="Reconciler error" 
"error"="error creating AzureManagedControlPlane capz-e2e-t4o6mu/capz-e2e-t4o6mu-aks: failed to reconcile AzureManagedControlPlane service 
managedcluster: failed to create resource capz-e2e-t4o6mu-aks/capz-e2e-t4o6mu-aks (service: managedcluster): 
Code=\"VMCannotFitEphemeralOSDisk\" 
Message=\"The virtual machine size Standard_B2s has a cache size of 32212254720 bytes and temporary disk size of 8589934592 bytes, but the OS disk requires 42949672960 bytes. 
  Use a VM size with larger cache, larger temp disk, or disable ephemeral OS.\"" 
"azureManagedControlPlane"={"name":"capz-e2e-t4o6mu-aks","namespace":"capz-e2e-t4o6mu"} "controller"="azuremanagedcontrolplane" "controllerGroup"="infrastructure.cluster.x-k8s.io" 

[CecileRobertMichon]

So close, but AKS doesn't like it

we could use a separate VM size for ephemeral OS scenarios

[nawazkh]

Ooh cost saving, nice!

we could use a separate VM size for ephemeral OS scenarios

How do we achieve handpicking VMs in "ephemeral OS scenarios" for testing in CI ? Via separate configs?
Also, what is implied by size in the statement? Is it the VM's memory, storage, or cache? Or all of it?

[mboersma]

How do we achieve handpicking VMs in "ephemeral OS scenarios" for testing in CI ?

It's a good question. I'm not sure yet. I'm hoping just to override AZURE_NODE_MACHINE_TYPE for the relevant test specs, but use Standard_B2s where we can.

[mboersma]

/hold cancel

The CI Azure subscription appears to have generous quota for Standard BS family, equal to or better than it has for Standard DS Family.

[jackfrancis]

/lgtm
/approve

Thanks for sticking with this @mboersma!

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jackfrancis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jackfrancis]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mboersma]

Should we also effectively recommend that users start with Standard_B2s? It seems to handle our e2e workloads, but I don't know whether real-world use might find it lacking.

[jackfrancis]

I'm not sure that the current existence of Standard_D2s_v3 is anything more than a "here's a known-working, barely operational example" for a VM SKU.

In other words, we have a long history of including in our reference configs the VM SKU that we use for testing — which includes in its positive criteria "cost-efficient" — (AFAICT this is the reason we use Standard_D2s_v3, not that it's actually a SKU we would recommend for real-world production use) so I don't think this amounts to a significant change.

[mboersma]

Makes sense to me, thanks for the thoughtful comment.

Also, it's really in the Cluster API Book Quick Start that we say anything that could be interpreted as a recommendation:

# Select VM types.
export AZURE_CONTROL_PLANE_MACHINE_TYPE="Standard_D2s_v3"
export AZURE_NODE_MACHINE_TYPE="Standard_D2s_v3"

and that's not going to change with this PR. So I guess this doesn't rock the boat that badly.

[mboersma]

/hold

Just had a question above about whether Standard_B2s should be in docs, or just in our e2e configs.

[mboersma]

/hold cancel

[nawazkh]

/lgtm++

[mboersma]

/retest-required

Looks like a flake where 2/3 nodes came up, which I've seen outside this PR.