fix: awsmachinepool use subnet filters

[calvix]

What type of PR is this?

/kind bug

What this PR does / why we need it:
implements was subnet lookup specified by filters in AWSMachinePool.Spec.Subnets

The previous implementation only worked with IDs, but now users can specify filters to select specific subnets that will be used for the machinepool.

Fixes #2609

Special notes for your reviewer:

Checklist:

• squashed commits
• includes documentation
• adds unit tests

Release note:

Add implementation of subnets filters for AWSMachinePool CR

[k8s-ci-robot]

Welcome @calvix!

It looks like this is your first PR to kubernetes-sigs/cluster-api-provider-aws 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api-provider-aws has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @calvix. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[randomvariable]

/ok-to-test
/area networking
/priority important-soon

[randomvariable]

I'd prefer for this to be implemented everywhere there is a Subnet currently using a AWSResourceReference, rather than having this very ad hoc implementation in this one specific CRD. It doesn't really fix the API issue we have.

I'll try and include this in the API cleanup.

[calvix]

I'd prefer for this to be implemented everywhere there is a Subnet currently using a AWSResourceReference, rather than having this very ad hoc implementation in this one specific CRD. It doesn't really fix the API issue we have.

I'll try and include this in the API cleanup.

As far as I know, the only other resource that has reference to the subnet is AWSMachine and the machine controller already has the implementation for the filters (I used it as a template for this)

cluster-api-provider-aws/pkg/cloud/services/ec2/instances.go

Lines 293 to 320 in 327e9b8

	case scope.AWSMachine.Spec.Subnet != nil && scope.AWSMachine.Spec.Subnet.Filters != nil:
	criteria := []*ec2.Filter{
	filter.EC2.SubnetStates(ec2.SubnetStatePending, ec2.SubnetStateAvailable),
	}
	if !scope.IsExternallyManaged() {
	criteria = append(criteria, filter.EC2.VPC(s.scope.VPC().ID))
	}
	if failureDomain != nil {
	criteria = append(criteria, filter.EC2.AvailabilityZone(*failureDomain))
	}
	for _, f := range scope.AWSMachine.Spec.Subnet.Filters {
	criteria = append(criteria, &ec2.Filter{Name: aws.String(f.Name), Values: aws.StringSlice(f.Values)})
	}
	subnets, err := s.getFilteredSubnets(criteria...)
	if err != nil {
	return "", errors.Wrapf(err, "failed to filter subnets for criteria %q", criteria)
	}
	if len(subnets) == 0 {
	record.Warnf(scope.AWSMachine, "FailedCreate",
	"Failed to create instance: no subnets available matching filters %q", scope.AWSMachine.Spec.Subnet.Filters)
	return "", awserrors.NewFailedDependency(
	fmt.Sprintf("failed to run machine %q, no subnets available matching filters %q",
	scope.Name(),
	scope.AWSMachine.Spec.Subnet.Filters,
	),
	)
	}
	return *subnets[0].SubnetId, nil

[randomvariable]

In which case, can we dedupe the code and have it in a single plac?

[sedefsavas]

@calvix Can we try to implement a common method for all other subnets to use as @randomvariable pointed in the comment above?
We cannot have an API change but I think if we at least implement a common logic to handle filter field, it will be straightforward to fix the api in the v1beta2 release.

[k8s-ci-robot]

@calvix: Adding label do-not-merge/contains-merge-commits because PR contains merge commits, which are not allowed in this repository.
Use git rebase to reapply your commits on top of the target branch. Detailed instructions for doing so can be found here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign richardcase after the PR has been reviewed.
You can assign the PR to them by writing /assign @richardcase in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[AverageMarcus]

@sedefsavas @randomvariable Do you have any suggestions for how best to refactor this in a non-API-changing way?

The inconsistencies with AWSResourceReference have been long-standing and not yet addressed in any of the API changes so would hate to have this blocked on that.

AWAMachinePool and AWSMachine are (as far as I can see) the only two types that use AWSResourceReference for defining subnets. The former as a slice, the later as a single pointer, so the logic between the two isn't exactly the same so I'm not sure a single function even makes sense for this.

[calvix]

@sedefsavas @randomvariable I also think the logic between the two isn't exactly the same so I'm not sure a single function even makes sense for this. Unless you have some good suggestions on how to do that.

[shivi28]

Thank you so much @calvix for raising this PR and doing initial research/discussion to resolve #2609. 🎉

And due to priority reasons, I also have raised a PR to implement the remaining logic and feeling happy to add you as a co-author :)

[Ankitasw]

/close
in favor of #3255

[k8s-ci-robot]

@Ankitasw: Closed this PR.

In response to this:

/close
in favor of #3255

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[calvix]

thanks for sorting it out :)