Use multiple zones in case of multiple subnets

[Shilpa-Gokul]

What this PR does / why we need it:
Currently when there are multiple subnets specified, it uses the first available vpcZone for all subnets by default and as a result, there is a clash in the CIDR. Hence added logic to use the subsequent vpcZones for the subnets.

If there are multiple subnets provided than the available vpcZones, the zones are selected based on round robin method.

This PR contains change to use vpcSubnetIPAddressCount, where when two subnets are to be created in the same zone, the subnets are created with the next available address prefix based on the vpcSubnetIPAddressCount.
Taking one of the below examples for reference

- name: subnet-1
    zone: eu-es-2
  - name: subnet-2
  - name: subnet-3
  - name: subnet-4
  - name: subnet-5
    zone: eu-es-2
  - name: subnet-6
    zone: eu-es-1
  - name: subnet-7

The zone selection is done as below, considering index 0 -> Madrid 1, index 1 -> Madrid 2, index 2 -> Madrid 3

subnet 1 -> Madrid 2 (since zone is mentioned)
subnet 2 -> Madrid 2 (since index is 1)
subnet 3 -> Madrid 3 (since index is 2)
subnet 4 -> Madrid 1 (index % len(totalvpcZones)= 3%3= 0 so Madrid 1)
subnet 5 -> Madrid 2 (since zone is mentioned)
subnet 6 -> Madrid 1 (since zone is mentioned)
subnet 7 -> Madrid 1 (index % len(totalvpcZones)= 6%3= 0 so Madrid 1)

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1779

Special notes for your reviewer:

/area provider/ibmcloud

1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

[k8s-ci-robot]

Hi @Shilpa-Gokul. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[netlify]

✅ Deploy Preview for kubernetes-sigs-cluster-api-ibmcloud ready!

Name	Link
🔨 Latest commit	b4bd92b
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-cluster-api-ibmcloud/deploys/6746d0f0f736e200086aa65a
😎 Deploy Preview	https://deploy-preview-1793--kubernetes-sigs-cluster-api-ibmcloud.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[Karthik-K-N]

How about handling this in webhooks, Having a default webhook which adds unique zone to subnets if it does not present.
Also I think we can error out if the user specifies more subnets than zone.

@dharaneeshvrd will there be any uses cases of user setting more subnets? like vpc region has 2 zones but user wants to create 4 subnets?

[dharaneeshvrd]

will there be any uses cases of user setting more subnets? like vpc region has 2 zones but user wants to create 4 subnets?

its possible, we can just round robin the zones once subnets created for all zones.
i.e. consider the region has 2 zones, 10.241.0.0/24 is already used for subnet 1 for subnet 3, it automatically picks 10.241.1.0/24.

[Shilpa-Gokul]

Tested the following scenarios

Case 1: When user provides more subnets than the available vpczones and provides no zone information with each subnet (subnets are created in the available vpc zones in a round robin method)

vpcSubnets:
  - name: capi-test-cluster-sg-vpcsubnet
  - name: capi-test-cluster-sg-vpcsubnet-1
  - name: capi-test-cluster-sg-vpcsubnet-2
  - name: capi-test-cluster-sg-vpcsubnet-3
  - name: capi-test-cluster-sg-vpcsubnet-4
  - name: capi-test-cluster-sg-vpcsubnet-5
  - name: capi-test-cluster-sg-vpcsubnet-6
  zone: mad02

Case 2: When user provides 2 subnets in the same zone to be created

 vpcSubnets:
  - name: capi-test-cluster-sg-vpcsubnet-1
    zone: eu-es-2
  - name: capi-test-cluster-sg-vpcsubnet-2
    zone: eu-es-2

Case 3: User provides 1 subnet and the zone to be created in

 vpcSubnets:
  - name: capi-test-cluster-sg-vpcsubnet-1
    zone: eu-es-2

Case 4: When user provides already created subnet

vpc:
  name: capi-test-cluster-sg-vpc-new
  region: eu-es
vpcSubnets:
- name: sn-20240723-01
zone: mad02

Log- [manager] I0723 12:15:38.121366 1 powervs_cluster.go:1103] "VPC subnet ID is set, fetching details" controller="ibmpowervscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="IBMPowerVSCluster" IBMPowerVSCluster="default/capi-test-cluster-sg" namespace="default" name="capi-test-cluster-sg" reconcileID="19a61637-59a3-4ea0-896f-6bb3a4616902" cluster="default/capi-test-cluster-sg" id="02w7-e123549e-4414-480c-b018-7017492889b3"

Case 5: When user provides subnets with no zone information

vpcSubnets:
  - name: capi-test-cluster-sg-vpcsubnet-1
  - name: capi-test-cluster-sg-vpcsubnet-2
  zone: mad02

@Karthik-K-N @Amulyam24 Please check this and let me know if I have to test anything else

[dharaneeshvrd]

What happens to this case?

vpcSubnets:
  - name: subnet-1
    zone: eu-es-2
  - name: subnet-2

Again it will get a collision I guess, since we are not handling the combination of both user provided and not provided.
Please handle this as well and if you can add UT with different corner cases that would be good for ReconcileVPCSubnets.

[Shilpa-Gokul]

Currently when we use vpcSubnetIPAddressCount, when two subnets are to be created in the same zone, the subnets are created with the next available address prefix based on the vpcSubnetIPAddressCount.
When no zones are mentioned, the subnets are assigned to the zones in round robin method.
Taking one of the below examples for reference

- name: subnet-1
    zone: eu-es-2
  - name: subnet-2
  - name: subnet-3
  - name: subnet-4
  - name: subnet-5
    zone: eu-es-2
  - name: subnet-6
    zone: eu-es-1
  - name: subnet-7

The zone selection is done as below, considering index 0 -> Madrid 1, index 1 -> Madrid 2, index 2 -> Madrid 3

subnet 1 -> Madrid 2 (since zone is mentioned)
subnet 2 -> Madrid 2 (since index is 1)
subnet 3 -> Madrid 3 (since index is 2)
subnet 4 -> Madrid 1 (index % len(totalvpcZones)= 3%3= 0 so Madrid 1)
subnet 5 -> Madrid 2 (since zone is mentioned)
subnet 6 -> Madrid 1 (since zone is mentioned)
subnet 7 -> Madrid 1 (index % len(totalvpcZones)= 6%3= 0 so Madrid 1)

Tested the following scenarios.

Usecase 1: When user provides more subnets than the available VPC zones and also mentions zone for one subnet

vpcSubnets:
  - name: subnet-1
    zone: eu-es-2
  - name: subnet-2
  - name: subnet-3
  - name: subnet-4
  - name: subnet-5
zone: mad02

Usecase 2: When user provides subnets more than the available VPC zones and mentions zone randomly for few subnets

vpcSubnets:
- name: subnet-1
  zone: eu-es-2
- name: subnet-2
- name: subnet-3
- name: subnet-4
- name: subnet-5
  zone: eu-es-2
- name: subnet-6
  zone: eu-es-1
- name: subnet-7
zone: mad02

Usecase 3: When user provides subnets less than the available VPC zones along with zone specified for each subnet

vpcSubnets:
  - name: subnet-1
    zone: eu-es-2
  - name: subnet-2
    zone: eu-es-1
zone: mad02

Usecase 4: When user provides already existing subnet

 vpcSubnets:
  - name: eu-es-1-default-subnet
    zone: eu-es-1
 zone: mad02

I1125 13:27:02.827412 14 powervs_cluster.go:1167] "Found VPC subnet in IBM Cloud" controller="ibmpowervscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="IBMPowerVSCluster" IBMPowerVSCluster="default/capi-test-cluster-sg" namespace="default" name="capi-test-cluster-sg" reconcileID="20586ea8-2019-4770-97bd-81cd925624f4" cluster="default/capi-test-cluster-sg" subnetID="02w7-5e7172ea-8306-444f-b4e6-97acbf35f530"

Usecase 5: When user specifies subnets more than the available VPC zones with no zones mentioned

 vpcSubnets:
  - name: subnet1
  - name: subnet2
  - name: subnet3
  - name: subnet4
  - name: subnet5
  - name: subnet6
zone: mad02

Usecase 6: When user doesn't provide any subnets explicitly, a subnet is created in each available VPC zone

 vpc:
    name: capi-test-cluster-sg-vpc
    region: eu-es
  zone: mad02

[dharaneeshvrd]

Can you please add a UT for index rotating logic as well?

[Karthik-K-N]

This has been one of the difficult feature to implement and it has gone through lots of iteration during implementation as well, Thanks @Shilpa-Gokul for take care of this. Appreciate your commitment and ownership on this feature.

/lgtm
/assign @dharaneeshvrd

[dharaneeshvrd]

Just few nits.
Overall LGTM!

[Shilpa-Gokul]

/test pull-cluster-api-provider-ibmcloud-coverage

[dharaneeshvrd]

/lgtm

[Prajyot-Parab]

/lgtm
Thanks @Shilpa-Gokul

[Prajyot-Parab]

/hold for @Amulyam24 review

[Amulyam24]

/unhold

Thank you @Shilpa-Gokul!

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Amulyam24, Prajyot-Parab, Shilpa-Gokul

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Prajyot-Parab]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mkumatag]

Thanks @Shilpa-Gokul for fixing this, it was a tricky one to solve.

[Shilpa-Gokul]

@mkumatag @Karthik-K-N @Amulyam24 @Prajyot-Parab @dharaneeshvrd Thanks to you all for helping me understand the codebase, different usecases and scenarios