Install cloud-provider-azure chart in e2e tests using CAAPH #3422

Jont828 · 2023-04-13T20:31:23Z

What type of PR is this?
/kind feature

What this PR does / why we need it: Use CAAPH to install cloud-provider-azure chart instead of Helm SDK in e2e tests.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #3736

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

squashed commits
includes documentation
adds unit tests

Release note:

Install cloud-provider-azure Helm chart using CAAPH

Jont828 · 2023-04-18T20:55:32Z

/retest

Jont828 · 2023-04-19T22:53:53Z

/retest

mboersma · 2023-08-16T17:22:33Z

/hold

I vendored CAAPH from the current main branch SHA and rebased, but we should update this to use the next CAAPH release tag.

codecov · 2023-08-16T17:42:04Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (220fe0f) 56.00% compared to head (a0aee60) 56.02%.
Report is 237 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3422      +/-   ##
==========================================
+ Coverage   56.00%   56.02%   +0.01%     
==========================================
  Files         190      190              
  Lines       19472    19473       +1     
==========================================
+ Hits        10906    10909       +3     
+ Misses       7948     7944       -4     
- Partials      618      620       +2

Files	Coverage Δ
internal/test/env/env.go	`53.73% <100.00%> (+0.70%)`	⬆️

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Jont828 · 2023-08-17T23:47:07Z

test/e2e/capi_test.go

@@ -241,6 +241,7 @@ var _ = Describe("Running the Cluster API E2E tests", func() {
 						InitWithBootstrapProviders:      []string{"kubeadm:v1.0.5"},
 						InitWithControlPlaneProviders:   []string{"kubeadm:v1.0.5"},
 						InitWithInfrastructureProviders: []string{"azure:v1.0.2"},
+						// Note: this probably won't work with CAAPH since the Helm install only works with CAPI/clusterctl v1.5.0.


@willie-yao I saw you made some changes here, I don't think we can use the CAAPH install with CAPI v1.0.5. Is there any way we can get around this?

Is there a way to add an exception to not use CAAPH for this test? This test is testing the upgrade from the earliest stable version of CAPI to current, so it will be useful to keep around. If not, is there a minimum version of CAPI that will work with CAAPH install?

mboersma · 2023-08-18T12:47:32Z

Focusing just on the pull-cluster-api-provider-azure-capi-e2e, it's interesting that only certain specs failed waiting for Calico to initialize. I ran a simple scenario locally with this branch and it passed:

AZWI=false GINKGO_FOCUS="Creating a highly available cluster \[REQUIRED\]" \ 
  SKIP_CLEANUP=true ./scripts/ci-e2e.sh

I poked at the workload cluster during this test, and cloud-provider-azure components were all installed and running. So CAAPH seems basically to be working, but somehow this PR makes Calico fail some of the time...

Jont828 · 2023-08-18T21:32:39Z

Yeah, I ran the e2e tests locally as well and it seemed to work fine for me. I wonder if the version upgrade is causing any issues.

mboersma · 2023-08-25T15:38:03Z

test/e2e/azure_test.go

@@ -76,6 +80,17 @@ var _ = Describe("Workload cluster creation", func() {
 		namespace, cancelWatches, err = setupSpecNamespace(ctx, clusterNamePrefix, bootstrapClusterProxy, artifactFolder)
 		Expect(err).NotTo(HaveOccurred())

+		By("Initialize bootstrap client to install add-ons")


Would it be feasible to detect the CAPI version here and somehow fall back to the old helm install behavior if it's not >= 1.5.0?

k8s-ci-robot · 2023-09-05T23:45:08Z

@Jont828: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-provider-azure-verify	`a0aee60`	link	true	`/test pull-cluster-api-provider-azure-verify`
pull-cluster-api-provider-azure-e2e-aks	`a0aee60`	link	true	`/test pull-cluster-api-provider-azure-e2e-aks`
pull-cluster-api-provider-azure-conformance	`a0aee60`	link	false	`/test pull-cluster-api-provider-azure-conformance`
pull-cluster-api-provider-azure-conformance-with-ci-artifacts	`a0aee60`	link	false	`/test pull-cluster-api-provider-azure-conformance-with-ci-artifacts`
pull-cluster-api-provider-azure-e2e	`a0aee60`	link	true	`/test pull-cluster-api-provider-azure-e2e`
pull-cluster-api-provider-azure-capi-e2e	`a0aee60`	link	false	`/test pull-cluster-api-provider-azure-capi-e2e`
pull-cluster-api-provider-azure-windows-containerd-upstream-with-ci-artifacts	`a0aee60`	link	false	`/test pull-cluster-api-provider-azure-windows-containerd-upstream-with-ci-artifacts`
pull-cluster-api-provider-azure-apiversion-upgrade	`a0aee60`	link	false	`/test pull-cluster-api-provider-azure-apiversion-upgrade`
pull-cluster-api-provider-azure-ci-entrypoint	`a0aee60`	link	true	`/test pull-cluster-api-provider-azure-ci-entrypoint`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot · 2023-09-28T11:52:59Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

CecileRobertMichon · 2023-10-31T23:20:37Z

/close

let's open a new PR for this change once it's ready

k8s-ci-robot · 2023-10-31T23:20:43Z

@CecileRobertMichon: Closed this PR.

In response to this:

/close

let's open a new PR for this change once it's ready

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Apr 13, 2023

k8s-ci-robot requested review from CecileRobertMichon and jsturtevant April 13, 2023 20:31

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 13, 2023

Jont828 changed the title ~~Install cloud-provider-azure chart in e2e tests using CAAPH~~ [WIP] Install cloud-provider-azure chart in e2e tests using CAAPH Apr 13, 2023

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 13, 2023

Jont828 force-pushed the caaph-e2e branch 2 times, most recently from e60bf36 to 5ccde2c Compare April 18, 2023 00:21

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 18, 2023

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 22, 2023

Jont828 force-pushed the caaph-e2e branch 4 times, most recently from 866fcfd to 7c385be Compare May 15, 2023 22:59

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels May 15, 2023

Jont828 changed the title ~~[WIP] Install cloud-provider-azure chart in e2e tests using CAAPH~~ Install cloud-provider-azure chart in e2e tests using CAAPH May 15, 2023

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 15, 2023

Jont828 force-pushed the caaph-e2e branch from 7c385be to 2970702 Compare May 15, 2023 23:07

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 15, 2023

Jont828 force-pushed the caaph-e2e branch 2 times, most recently from b6452f3 to 580565e Compare May 17, 2023 01:43

Jont828 mentioned this pull request May 17, 2023

Install calico chart in e2e tests using CAAPH #3571

Closed

4 tasks

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Aug 14, 2023

mboersma force-pushed the caaph-e2e branch from 9c6f667 to 121b797 Compare August 16, 2023 17:21

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 16, 2023

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 16, 2023

Jont828 force-pushed the caaph-e2e branch 2 times, most recently from 8957842 to d7492d1 Compare August 16, 2023 22:24

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 16, 2023

Jont828 commented Aug 17, 2023

View reviewed changes

mboersma reviewed Aug 25, 2023

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 30, 2023

Install cloud-provider-azure Helm chart in e2e tests using CAAPH

97fd3f1

Jont828 force-pushed the caaph-e2e branch from 5d8eefd to 97fd3f1 Compare September 5, 2023 19:26

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 5, 2023

Add cloud-provider-azure HCP to prow templates

2c485a3

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 5, 2023

Jont828 added 2 commits September 5, 2023 19:11

Remove in tree and fix private cluster

c0350d1

nuke entrypoint functions :)

a0aee60

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 28, 2023

k8s-ci-robot closed this Oct 31, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Install cloud-provider-azure chart in e2e tests using CAAPH #3422

Install cloud-provider-azure chart in e2e tests using CAAPH #3422

Jont828 commented Apr 13, 2023 •

edited

Loading

Jont828 commented Apr 18, 2023

Jont828 commented Apr 19, 2023

mboersma commented Aug 16, 2023

codecov bot commented Aug 16, 2023 •

edited

Loading

Jont828 Aug 17, 2023

willie-yao Aug 21, 2023

mboersma commented Aug 18, 2023

Jont828 commented Aug 18, 2023

mboersma Aug 25, 2023

k8s-ci-robot commented Sep 5, 2023

k8s-ci-robot commented Sep 28, 2023

CecileRobertMichon commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023

Install cloud-provider-azure chart in e2e tests using CAAPH #3422

Install cloud-provider-azure chart in e2e tests using CAAPH #3422

Conversation

Jont828 commented Apr 13, 2023 • edited Loading

Jont828 commented Apr 18, 2023

Jont828 commented Apr 19, 2023

mboersma commented Aug 16, 2023

codecov bot commented Aug 16, 2023 • edited Loading

Codecov Report

Jont828 Aug 17, 2023

Choose a reason for hiding this comment

willie-yao Aug 21, 2023

Choose a reason for hiding this comment

mboersma commented Aug 18, 2023

Jont828 commented Aug 18, 2023

mboersma Aug 25, 2023

Choose a reason for hiding this comment

k8s-ci-robot commented Sep 5, 2023

k8s-ci-robot commented Sep 28, 2023

CecileRobertMichon commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023

Jont828 commented Apr 13, 2023 •

edited

Loading

codecov bot commented Aug 16, 2023 •

edited

Loading