Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update custom-builds.yaml template to UAMI #4141

Conversation

DannyBrito
Copy link
Contributor

@DannyBrito DannyBrito commented Oct 16, 2023

What type of PR is this?
/kind feature

What this PR does / why we need it:
Update custom-builds.yaml template components to use UAMI and add az cli for binary replacement download to enable AUTH in azure storage account via UAMI.

cc: @jeremyrickard

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

  • cherry-pick candidate

TODOs:

  • squashed commits
  • includes documentation
  • adds unit tests

Release note:


… and az storage blob download

Signed-off-by: Jeremy Rickard <[email protected]>

Co-authored-by: Danny Brito <[email protected]>
@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 16, 2023
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Oct 16, 2023
@k8s-ci-robot
Copy link
Contributor

Hi @DannyBrito. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Oct 16, 2023
@mboersma
Copy link
Contributor

mboersma commented Oct 16, 2023

I assume for this to work, we need kubernetes-sigs/image-builder#1326 to merge, and then to build new CAPZ reference images for e2e to test with, right?

Edit: nope, this stands on its own and the image-builder PR will eventually simplify it by handling az install.

@mboersma
Copy link
Contributor

/release-note-none

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Oct 16, 2023
@codecov
Copy link

codecov bot commented Oct 16, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (620382a) 57.83% compared to head (a0702cd) 57.82%.
Report is 32 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4141      +/-   ##
==========================================
- Coverage   57.83%   57.82%   -0.02%     
==========================================
  Files         187      187              
  Lines       19195    19195              
==========================================
- Hits        11101    11099       -2     
- Misses       7466     7468       +2     
  Partials      628      628              

see 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@CecileRobertMichon
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 16, 2023
@CecileRobertMichon
Copy link
Contributor

/retest

flake tracked in #4123

Copy link
Contributor

@CecileRobertMichon CecileRobertMichon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have a presubmit for this template in CAPZ so I opened kubernetes/test-infra#31045 to add one so we don't regress k/k tests

@mboersma
Copy link
Contributor

/retest

Network connectivity flake AFAICT.

@jackfrancis
Copy link
Contributor

/assign

@CecileRobertMichon
Copy link
Contributor

/test ls

@k8s-ci-robot
Copy link
Contributor

@CecileRobertMichon: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test pull-cluster-api-provider-azure-build
  • /test pull-cluster-api-provider-azure-ci-entrypoint
  • /test pull-cluster-api-provider-azure-e2e
  • /test pull-cluster-api-provider-azure-e2e-aks
  • /test pull-cluster-api-provider-azure-test
  • /test pull-cluster-api-provider-azure-verify

The following commands are available to trigger optional jobs:

  • /test pull-cluster-api-provider-azure-apidiff
  • /test pull-cluster-api-provider-azure-apiversion-upgrade
  • /test pull-cluster-api-provider-azure-capi-e2e
  • /test pull-cluster-api-provider-azure-conformance
  • /test pull-cluster-api-provider-azure-conformance-custom-builds
  • /test pull-cluster-api-provider-azure-conformance-dual-stack-with-ci-artifacts
  • /test pull-cluster-api-provider-azure-conformance-ipv6-with-ci-artifacts
  • /test pull-cluster-api-provider-azure-conformance-with-ci-artifacts
  • /test pull-cluster-api-provider-azure-e2e-optional
  • /test pull-cluster-api-provider-azure-e2e-workload-upgrade
  • /test pull-cluster-api-provider-azure-windows-containerd-upstream-with-ci-artifacts
  • /test pull-cluster-api-provider-azure-windows-containerd-upstream-with-ci-artifacts-serial-slow

Use /test all to run the following jobs that were automatically triggered:

  • pull-cluster-api-provider-azure-apidiff
  • pull-cluster-api-provider-azure-build
  • pull-cluster-api-provider-azure-capi-e2e
  • pull-cluster-api-provider-azure-ci-entrypoint
  • pull-cluster-api-provider-azure-conformance
  • pull-cluster-api-provider-azure-conformance-custom-builds
  • pull-cluster-api-provider-azure-conformance-dual-stack-with-ci-artifacts
  • pull-cluster-api-provider-azure-conformance-ipv6-with-ci-artifacts
  • pull-cluster-api-provider-azure-conformance-with-ci-artifacts
  • pull-cluster-api-provider-azure-e2e
  • pull-cluster-api-provider-azure-e2e-aks
  • pull-cluster-api-provider-azure-test
  • pull-cluster-api-provider-azure-verify
  • pull-cluster-api-provider-azure-windows-containerd-upstream-with-ci-artifacts

In response to this:

/test ls

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@CecileRobertMichon
Copy link
Contributor

/test pull-cluster-api-provider-azure-conformance-custom-builds

@CecileRobertMichon
Copy link
Contributor

/retest

looks like the actual cluster build succeeded

@CecileRobertMichon
Copy link
Contributor

/hold

@DannyBrito I'm seeing the following error in the custom-builds job cloud-init logs:

[2023-10-20 01:16:16] * installing package: kubeadm v1.29.0-alpha.2.379+88d9573c3033af
[2023-10-20 01:16:22] ERROR: 
[2023-10-20 01:16:22] You do not have the required permissions needed to perform this operation.
[2023-10-20 01:16:22] Depending on your operation, you may need to be assigned one of the following roles:
[2023-10-20 01:16:22]     "Storage Blob Data Owner"
[2023-10-20 01:16:22]     "Storage Blob Data Contributor"
[2023-10-20 01:16:22]     "Storage Blob Data Reader"
[2023-10-20 01:16:22]     "Storage Queue Data Contributor"
[2023-10-20 01:16:22]     "Storage Queue Data Reader"
[2023-10-20 01:16:22]     "Storage Table Data Contributor"
[2023-10-20 01:16:22]     "Storage Table Data Reader"
[2023-10-20 01:16:22] 
[2023-10-20 01:16:22] If you want to use the old authentication method and allow querying for the right account key, please use the "--auth-mode" parameter and "key" value.

https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cluster-api-provider-azure/4141/pull-cluster-api-provider-azure-conformance-custom-builds/1715164798053781504/artifacts/clusters/capz-conf-6ps1ot/machines/capz-conf-6ps1ot-md-0-57lzx-8dl6m/cloud-init-output.log

Seems like something needs to be setup on the prow subscription managed identity

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 20, 2023
@DannyBrito
Copy link
Contributor Author

/test pull-cluster-api-provider-azure-conformance-custom-builds

@CecileRobertMichon
Copy link
Contributor

Created kubernetes/test-infra#31106 to add a windows job

@DannyBrito
Copy link
Contributor Author

@CecileRobertMichon, the ci custom job passed, and control-plane cloud-int logs look like it was able to pull binaries after role assignation to UAMI, and kubelet version is replaced in the nodes with k8s alpha version node-describe.txt

@CecileRobertMichon
Copy link
Contributor

@DannyBrito perfect. Just waiting for a review on the test-infra PR above to add a windows job so we can test this change for windows too.

@DannyBrito
Copy link
Contributor Author

/test pull-cluster-api-provider-azure-windows-containerd-upstream-custom-builds

@CecileRobertMichon
Copy link
Contributor

ERROR: Operation returned an invalid status 'The specified resource name length is not within the permissible limits.'
ErrorCode:OutOfRangeInput

Looks like we may be running into #3673

@CecileRobertMichon
Copy link
Contributor

Opened #4172 to fix it

@CecileRobertMichon
Copy link
Contributor

Now that kubernetes/test-infra#31126 merged, windows job should be able to run

/test pull-cluster-api-provider-azure-windows-custom-builds

@k8s-ci-robot
Copy link
Contributor

@CecileRobertMichon: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test pull-cluster-api-provider-azure-build
  • /test pull-cluster-api-provider-azure-ci-entrypoint
  • /test pull-cluster-api-provider-azure-e2e
  • /test pull-cluster-api-provider-azure-e2e-aks
  • /test pull-cluster-api-provider-azure-test
  • /test pull-cluster-api-provider-azure-verify

The following commands are available to trigger optional jobs:

  • /test pull-cluster-api-provider-azure-apidiff
  • /test pull-cluster-api-provider-azure-apiversion-upgrade
  • /test pull-cluster-api-provider-azure-capi-e2e
  • /test pull-cluster-api-provider-azure-conformance
  • /test pull-cluster-api-provider-azure-conformance-custom-builds
  • /test pull-cluster-api-provider-azure-conformance-dual-stack-with-ci-artifacts
  • /test pull-cluster-api-provider-azure-conformance-ipv6-with-ci-artifacts
  • /test pull-cluster-api-provider-azure-conformance-with-ci-artifacts
  • /test pull-cluster-api-provider-azure-e2e-optional
  • /test pull-cluster-api-provider-azure-e2e-workload-upgrade
  • /test pull-cluster-api-provider-azure-windows--custom-builds
  • /test pull-cluster-api-provider-azure-windows-containerd-upstream-with-ci-artifacts-serial-slow
  • /test pull-cluster-api-provider-azure-windows-with-ci-artifacts

Use /test all to run the following jobs that were automatically triggered:

  • pull-cluster-api-provider-azure-apidiff
  • pull-cluster-api-provider-azure-build
  • pull-cluster-api-provider-azure-capi-e2e
  • pull-cluster-api-provider-azure-ci-entrypoint
  • pull-cluster-api-provider-azure-conformance
  • pull-cluster-api-provider-azure-conformance-custom-builds
  • pull-cluster-api-provider-azure-conformance-dual-stack-with-ci-artifacts
  • pull-cluster-api-provider-azure-conformance-ipv6-with-ci-artifacts
  • pull-cluster-api-provider-azure-conformance-with-ci-artifacts
  • pull-cluster-api-provider-azure-e2e
  • pull-cluster-api-provider-azure-e2e-aks
  • pull-cluster-api-provider-azure-test
  • pull-cluster-api-provider-azure-verify
  • pull-cluster-api-provider-azure-windows--custom-builds
  • pull-cluster-api-provider-azure-windows-with-ci-artifacts

In response to this:

Now that kubernetes/test-infra#31126 merged, windows job should be able to run

/test pull-cluster-api-provider-azure-windows-custom-builds

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@CecileRobertMichon
Copy link
Contributor

/test pull-cluster-api-provider-azure-windows--custom-builds

@CecileRobertMichon
Copy link
Contributor

/test pull-cluster-api-provider-azure-windows-custom-builds

@CecileRobertMichon
Copy link
Contributor

/override pull-cluster-api-provider-azure-windows--custom-builds

job was renamed to fix typo

@k8s-ci-robot
Copy link
Contributor

@CecileRobertMichon: Overrode contexts on behalf of CecileRobertMichon: pull-cluster-api-provider-azure-windows--custom-builds

In response to this:

/override pull-cluster-api-provider-azure-windows--custom-builds

job was renamed to fix typo

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Oct 25, 2023

@DannyBrito: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cluster-api-provider-azure-windows-containerd-upstream-custom-builds a0702cd link false /test pull-cluster-api-provider-azure-windows-containerd-upstream-custom-builds
pull-cluster-api-provider-azure-windows--custom-builds a0702cd link false /test pull-cluster-api-provider-azure-windows--custom-builds

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@CecileRobertMichon
Copy link
Contributor

/retest

hoping that was a flake, but let's see...

@CecileRobertMichon
Copy link
Contributor

Windows custom build looks good as well

  Kubelet Version:            v1.29.0-alpha.2.560+af52a7052b3cf5-dirty
  Kube-Proxy Version:         v1.29.0-alpha.2.560+af52a7052b3cf5-dirty

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 26, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 73b50abe3e6144fec4bbc136872da7a5f9f3a8f9

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 26, 2023
@CecileRobertMichon
Copy link
Contributor

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 26, 2023
@k8s-ci-robot k8s-ci-robot merged commit 9a53ffb into kubernetes-sigs:main Oct 26, 2023
31 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.12 milestone Oct 26, 2023
@CecileRobertMichon
Copy link
Contributor

/cherry-pick release-1.11

@k8s-infra-cherrypick-robot

@CecileRobertMichon: new pull request created: #4202

In response to this:

/cherry-pick release-1.11

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

7 participants