Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AKS: enable isVnetManaged, add caching #2543

Merged
merged 1 commit into from
Aug 24, 2022
Merged

AKS: enable isVnetManaged, add caching #2543

merged 1 commit into from
Aug 24, 2022

Conversation

jackfrancis
Copy link
Contributor

@jackfrancis jackfrancis commented Aug 10, 2022 •
edited
Loading

What type of PR is this?

/kind bug
/kind feature

What this PR does / why we need it:

This PR enables the IsVnetManaged() method for AKS scenarios, which results in actual GET calls against the Azure API to determine in real-time whether the expected capz owner tag (tag is applied one time during cluster create) is present on the Virtual Network.

This will address the subnet delete issue during custom VNET scenarios.

Additionally, in order to account for additional calls against the Azure APIs we've implemented a local cache to ensure we don't call the Azure APIs more than once for VNET information per reconciliation.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2484

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

  • squashed commits
  • includes documentation
  • adds unit tests

Release note:

AKS: enable isVnetManaged, add caching

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/bug Categorizes issue or PR as related to a bug. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 10, 2022
@jackfrancis jackfrancis force-pushed the subnet-vnet-ismanaged-ratelimit branch from a2afb14 to beb627f Compare August 10, 2022 17:44
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 10, 2022
CecileRobertMichon
CecileRobertMichon reviewed Aug 10, 2022
View reviewed changes
Makefile Outdated Show resolved Hide resolved
@jackfrancis jackfrancis force-pushed the subnet-vnet-ismanaged-ratelimit branch from beb627f to f5d0484 Compare August 12, 2022 12:57
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 12, 2022
@jackfrancis jackfrancis force-pushed the subnet-vnet-ismanaged-ratelimit branch 2 times, most recently from 3239239 to fd1eb5c Compare August 15, 2022 20:22
@CecileRobertMichon CecileRobertMichon mentioned this pull request Aug 18, 2022
Closed
3 tasks
@jackfrancis jackfrancis force-pushed the subnet-vnet-ismanaged-ratelimit branch 3 times, most recently from 8b7ae51 to c02d048 Compare August 18, 2022 20:20
@mtougeron
Copy link
Member

Not sure if this is relevant, but should the commit message also be changed to be more relevant to what the PR is actually doing now? I'm not sure if that's being too picky in reviews...

@jackfrancis
Copy link
Contributor Author

Not sure if this is relevant, but should the commit message also be changed to be more relevant to what the PR is actually doing now? I'm not sure if that's being too picky in reviews...

Yes, will definitely change the title!

@jackfrancis
Copy link
Contributor Author

/retitle AKS: enable isVnetManaged, add caching

@k8s-ci-robot k8s-ci-robot changed the title WIP AKS: enable IsVnetManaged with rate limiting and 429 retry AKS: enable isVnetManaged, add caching Aug 18, 2022
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 18, 2022
mtougeron
mtougeron reviewed Aug 19, 2022
View reviewed changes
azure/scope/managedcontrolplane.go
ctx := context.Background()
ctx, log, done := tele.StartSpanWithLogger(ctx, "scope.ManagedControlPlaneScope.IsVnetManaged")
defer done()
isManaged, err := virtualnetworks.New(s).IsManaged(ctx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In cluster.go you called this variable isVnetManaged.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's nothing mysterious here, I'm just preferring to use a lowercase local var to capture the response from the method name that we're calling.

azure/scope/managedcontrolplane.go Outdated Show resolved Hide resolved
azure/scope/cluster.go Outdated Show resolved Hide resolved
azure/services/backoff.go Outdated Show resolved Hide resolved
azure/services/virtualnetworks/virtualnetworks.go Outdated Show resolved Hide resolved
@jackfrancis jackfrancis force-pushed the subnet-vnet-ismanaged-ratelimit branch 2 times, most recently from d9b70b0 to 76e2d8a Compare August 19, 2022 18:41
CecileRobertMichon
CecileRobertMichon reviewed Aug 19, 2022
View reviewed changes
azure/scope/cluster.go
if s.cache.isVnetManaged != nil {
return to.Bool(s.cache.isVnetManaged)
}
isVnetManaged := s.Vnet().ID == "" || s.Vnet().Tags.HasOwned(s.ClusterName())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should change this behavior to be the same as managed clusters eventually, although I agree this is the least dangerous change for now since it preserves existing behavior

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was honestly a bit confused when implementing this for clusters, so glad to hear that suggestion.

I can throw up really quick PR after this one lands so that the scope of this PR leans a bit more towards managed clusters only.

CecileRobertMichon
CecileRobertMichon reviewed Aug 19, 2022
View reviewed changes
Copy link
Contributor

@CecileRobertMichon CecileRobertMichon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 19, 2022
@CecileRobertMichon
Copy link
Contributor

Additionally, in order to account for additional calls against the Azure APIs I've wired in a rate limiting + HTTP 429 retry solution (inspired by cloud-provider-azure).

Remove this from PR description?

@jackfrancis
Copy link
Contributor Author

Additionally, in order to account for additional calls against the Azure APIs I've wired in a rate limiting + HTTP 429 retry solution (inspired by cloud-provider-azure).

Remove this from PR description?

Done

@jackfrancis jackfrancis force-pushed the subnet-vnet-ismanaged-ratelimit branch from 76e2d8a to 0f321e4 Compare August 19, 2022 23:25
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 19, 2022
@jackfrancis
Copy link
Contributor Author

/retest

@jackfrancis
Copy link
Contributor Author

@mtougeron @CecileRobertMichon this is ready for another (hopefully final? 🤞) review round

CecileRobertMichon
CecileRobertMichon reviewed Aug 22, 2022
View reviewed changes
Copy link
Contributor

@CecileRobertMichon CecileRobertMichon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 22, 2022
mtougeron
mtougeron approved these changes Aug 24, 2022
View reviewed changes
Copy link
Member

@mtougeron mtougeron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me.

/approve

@CecileRobertMichon
Copy link
Contributor

/approve

@CecileRobertMichon
Copy link
Contributor

/cherry-pick release-1.4

@k8s-infra-cherrypick-robot
Copy link

@CecileRobertMichon: once the present PR merges, I will cherry-pick it on top of release-1.4 in a new PR and assign it to you.

In response to this:

/cherry-pick release-1.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon, mtougeron

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 24, 2022
@k8s-ci-robot k8s-ci-robot merged commit c93fbfb into kubernetes-sigs:main Aug 24, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.5 milestone Aug 24, 2022
@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Aug 24, 2022
Merged
@k8s-infra-cherrypick-robot
Copy link

@CecileRobertMichon: new pull request created: #2600

In response to this:

/cherry-pick release-1.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jackfrancis jackfrancis deleted the subnet-vnet-ismanaged-ratelimit branch August 24, 2022 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CecileRobertMichon CecileRobertMichon CecileRobertMichon left review comments

@mtougeron mtougeron mtougeron approved these changes

@alexeldeib alexeldeib Awaiting requested review from alexeldeib

@jsturtevant jsturtevant Awaiting requested review from jsturtevant

Assignees

@CecileRobertMichon CecileRobertMichon

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.5
5 participants
@jackfrancis @mtougeron @CecileRobertMichon @k8s-infra-cherrypick-robot @k8s-ci-robot