Implement GCPMachinePool using MIGs

[CecileRobertMichon]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]

Implement GCPMachinePool using Managed Instance Groups. The GCPMachinePool implementation should follow the same package / group structure as CAPI and be added to the exp package, the experimental feature package. The feature should be gated using the MachinePool flag used in CAPI.

See also: kubernetes-sigs/cluster-api-provider-azure#483 and CAPI Machine Pool Proposal

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Note: As noted below in comments, the kubernetes-sigs/cluster-api#7107 also be supported/implemented as part of this

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[CecileRobertMichon]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[detiber]

/lifecycle frozen
/help

[k8s-ci-robot]

@detiber:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/lifecycle frozen
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jayesh-srivastava]

Hey, would like to work on this. Can I assign it myself?

[jayesh-srivastava]

/assign

[evanfreed]

@jayesh-srivastava just curious if you are still working on this? Myself or another colleague of mine might have some cycles to pick this up if you need any help.

[AverageMarcus]

@CecileRobertMichon could we also add as a requirement to this ticket that (if actually merged) the externally-managed autoscaller annotation is also supported / implemented as part of this?

[CecileRobertMichon]

@AverageMarcus added a note in description

[AverageMarcus]

Perfect! Thank you!

[nsilve]

I have pinged @jayesh-srivastava via Slack and he told me that he is not able to work on this issue due to personal commitments. @evanfreed and myself are going to work on it unless there is any objection.

[nsilve]

Pending blockers:

• Enable Feature Gates #643
• feat: add managedinstancegroups GoogleCloudPlatform/k8s-cloud-provider#71
• ✨ MachinePool annotation for externally managed autoscaler cluster-api#7107

[evanfreed]

/assign @nsilve

[evanfreed]

/unassign @jayesh-srivastava

[nsilve]

Azure does not support any kind of templating for instance creation, so all the configuration should be included into VMSS (so into AzureMachinePool). But both AWS and GCP have discrete template resources for ASG (launch template)/MIG (instance template) so we are having the following options:

1. manage GCP instance templates internally to GCP Machine Pool controller (like CAPA)
2. manage GCP instance templates via a separate GCP Instance Template controller
3. something else

Right now they are not managed at all since ControlPlane machines are not created via MIG, so GCPMachineTemplate configuration is passed directly to GCPMachine. Since no actual GCP Instance Template is created right now, we will need to find a solution for it if we proceed with solution 2 because we will need to create only the instance templates which are going to be used for workers created via MIG (although creating another instance template for Control Plane won't hurt anyone but it is going to be a unused resource).

Moreover, based on GCP documentation, instance templates can be used to create both virtual machine (VM) instances and managed instance groups (MIGs). That means that we could probably define GCPMachineTemplate and use them directly into GCPMachinePool (not tested)

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: GCPMachineTemplate
metadata:
  name: workload
spec:
  template:
    spec:
      image: projects/elastic-esp-dev/global/images/capi-ubuntu-2004-k8s-v1-24-1
      instanceType: n2-standard-2

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: GCPMachinePool
metadata:
  name: workload
spec:
  machineTemplate:
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: GCPMachineTemplate
      name: workload

Any ideas/comments/objections?

[nsilve]

/remove-lifecycle frozen

[nsilve]

Plans changed so I am afraid that I cannot proceed with that.

[nsilve]

/unassign @nsilve

[BrennenMM7]

/assign