✨ Implement Reconcile mode for ClusterResourceSet #7497

g-gaston · 2022-11-04T16:23:21Z

What this PR does / why we need it:

Implements Reconcile mode for ClusterResourceSet's, introduced by proposal #6555

k8s-ci-robot · 2022-11-04T16:23:30Z

Hi @g-gaston. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jonathanmeier5

This all sounds reasonable to me. Looking forward to using this new reconciliation strategy.

My understanding of this change is that we are adding a Reconcile resolution strategy to the Cluster Resource Set CRD. The new Reconcile strategy will update existing Secret and ConfigMap objects contents when they change in the CRS.

The bulk of this PRs changes business logic changes appear to come from the ApplyClusterResourceSet function. Some of its business logic is refactored into clusterresourceset_scope.go, where hashing and other utility functions are abstracted into the ResourceReconcileScope object.

Outside of the unit tests, how did you test this implementation?

exp/addons/api/v1beta1/clusterresourcesetbinding_types_test.go

exp/addons/internal/controllers/clusterresourceset_controller.go

exp/addons/api/v1beta1/clusterresourceset_types.go

g-gaston · 2022-11-08T23:01:33Z

This all sounds reasonable to me. Looking forward to using this new reconciliation strategy.

My understanding of this change is that we are adding a Reconcile resolution strategy to the Cluster Resource Set CRD. The new Reconcile strategy will update existing Secret and ConfigMap objects contents when they change in the CRS.

The bulk of this PRs changes business logic changes appear to come from the ApplyClusterResourceSet function. Some of its business logic is refactored into clusterresourceset_scope.go, where hashing and other utility functions are abstracted into the ResourceReconcileScope object.

Outside of the unit tests, how did you test this implementation?

The tests in clusterresourceset_controller_test.go are integration tests, then run the controller connected to a real etcd and api server (using envtest). So those are a bit more robust than just unit tests, the test the reconciliation logic end to end (or almost, since there is no a real workload cluster).

I also tested this manually running some capd clusters with tilt.

sbueringer

just a few nits, didn't get to the controllers folder yet

docs/book/src/tasks/experimental-features/cluster-resource-set.md

exp/addons/api/v1beta1/clusterresourceset_types.go

exp/addons/internal/controllers/clusterresourceset_controller.go

sbueringer · 2022-11-16T14:57:25Z

/retest

Co-authored-by: Stefan Büringer <[email protected]>

sbueringer

Thank you for working on this!!

Overall looks good to me, although I have to say I don't have a lot of experience with the CRS code base.

Would be nice to get some reviews from the other folks interested in this feature.

exp/addons/internal/controllers/clusterresourceset_controller.go

exp/addons/internal/controllers/clusterresourceset_helpers.go

exp/addons/internal/controllers/clusterresourceset_scope.go

exp/addons/internal/controllers/clusterresourceset_scope_test.go

Co-authored-by: Stefan Büringer <[email protected]>

exp/addons/internal/controllers/clusterresourceset_controller.go

exp/addons/internal/controllers/clusterresourceset_helpers.go

exp/addons/api/v1beta1/clusterresourcesetbinding_types_test.go

exp/addons/internal/controllers/clusterresourceset_controller.go

exp/addons/internal/controllers/clusterresourceset_helpers.go

sbueringer · 2023-01-20T12:26:59Z

exp/addons/internal/controllers/clusterresourceset_scope.go

+	// The create call is idempotent, so if the object already exists
+	// then do not consider it to be an error.


Yup Fabrizio is right.

There are two usages of createUnstructured. l.127 and l.168. In l.168 we check for already exists in l.127 we don't.

Or is that intentional? (that we ignore already exists in ApplyOnce and we surface it in Reconcile?)

exp/addons/internal/controllers/clusterresourceset_scope.go

Co-authored-by: Stefan Büringer <[email protected]>

Co-authored-by: Fabrizio Pandini <[email protected]>

sbueringer · 2023-02-03T15:08:24Z

/retest

sbueringer

Just a few nits, otherwise lgtm from my side (+ linter)

exp/addons/internal/controllers/clusterresourceset_controller_test.go

sbueringer · 2023-02-03T15:24:36Z

exp/addons/internal/controllers/clusterresourceset_controller_test.go

+		g.Expect(env.Update(ctx, testCluster)).To(Succeed())
+
+		t.Log("Updating the test config map with the missing namespace resource")
+		missingNamespace := randomNamespaceForTest(t)


As the linter is complaining about the way you're generating the name anyway. WDYT about just using something like this?

Suggested change

missingNamespace := randomNamespaceForTest(t)

missingNamespace := fmt.Sprintf("clusterresourceset-apply-once-%s", util.RandomString(6))

I can change it if you prefer but I thought it was a useful function to have.

btw fixed the linter

Fine for me to keep just seems more complicated to have our own random func here vs. using the util that we use in a lot of other places

(usually we also use a prefix for the namespace which roughly identifies the test, just makes it easier to map resources to tests when debugging envtest based tests)

Up to you

Co-authored-by: Stefan Büringer <[email protected]>

…t.go typo Co-authored-by: Stefan Büringer <[email protected]>

sbueringer · 2023-02-14T13:03:31Z

Thank you very much!

/lgtm

/assign @fabriziopandini

k8s-ci-robot · 2023-02-14T13:03:37Z

LGTM label has been added.

Git tree hash: cd1d00bb36ba52dc2f56564cb2ba15c199e378d7

fabriziopandini

great job
/lgtm
/approve

k8s-ci-robot · 2023-02-19T10:00:33Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [fabriziopandini]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 4, 2022

k8s-ci-robot requested review from CecileRobertMichon and stmcginnis November 4, 2022 16:23

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Nov 4, 2022

k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 4, 2022

jonathanmeier5 reviewed Nov 8, 2022

View reviewed changes

exp/addons/api/v1beta1/clusterresourcesetbinding_types_test.go Show resolved Hide resolved

exp/addons/internal/controllers/clusterresourceset_controller.go Outdated Show resolved Hide resolved

mitalipaygude reviewed Nov 8, 2022

View reviewed changes

exp/addons/api/v1beta1/clusterresourceset_types.go Show resolved Hide resolved

mitalipaygude approved these changes Nov 8, 2022

View reviewed changes

jonathanmeier5 approved these changes Nov 9, 2022

View reviewed changes

g-gaston mentioned this pull request Nov 9, 2022

Add patch for CAPI CRS Reconcile mode aws/eks-anywhere-build-tooling#1535

Merged

sbueringer reviewed Nov 16, 2022

View reviewed changes

docs/book/src/tasks/experimental-features/cluster-resource-set.md Show resolved Hide resolved

exp/addons/api/v1beta1/clusterresourceset_types.go Outdated Show resolved Hide resolved

exp/addons/internal/controllers/clusterresourceset_controller.go Outdated Show resolved Hide resolved

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 24, 2022

g-gaston and others added 5 commits November 28, 2022 18:51

Implement Reconcile mode for ClusterResourceSet

a6ef8dc

Update ApplyClusterResourceSet doc comment with Reconcile strategy

4e02c8f

Added strategy update instructions to docs

5f62bc7

Fix typo

b70f649

Co-authored-by: Stefan Büringer <[email protected]>

Use ResourceCreateOrUpdate to dry naked predicates

0c5ee40

g-gaston force-pushed the crs-reconcile branch from 7232186 to 0c5ee40 Compare November 28, 2022 18:51

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 28, 2022

sbueringer reviewed Nov 29, 2022

View reviewed changes

g-gaston and others added 2 commits December 5, 2022 17:35

Update exp/addons/internal/controllers/clusterresourceset_controller.go

9586cb5

Co-authored-by: Stefan Büringer <[email protected]>

Addressed review comment: minor cleanup

b0c7337

g-gaston requested review from sbueringer and removed request for stmcginnis, CecileRobertMichon and sbueringer December 5, 2022 23:50

g-gaston requested a review from fabriziopandini January 10, 2023 12:25

Implement top loevel apply methoid in resource scopes

6c656f8

g-gaston force-pushed the crs-reconcile branch from 74edeaf to 6c656f8 Compare January 10, 2023 12:28

fabriziopandini reviewed Jan 20, 2023

View reviewed changes

exp/addons/internal/controllers/clusterresourceset_controller.go Outdated Show resolved Hide resolved

exp/addons/internal/controllers/clusterresourceset_helpers.go Outdated Show resolved Hide resolved

sbueringer reviewed Jan 20, 2023

View reviewed changes

g-gaston and others added 4 commits January 25, 2023 16:45

Improve test case names

199fcce

Co-authored-by: Stefan Büringer <[email protected]>

Check scope error before setting binding to improve legibility

a69ab2e

Fix typo

d7d1ffa

Co-authored-by: Fabrizio Pandini <[email protected]>

Improve errors in crs normalize data and scope

2891a9c

g-gaston force-pushed the crs-reconcile branch from 57c87cc to 2891a9c Compare January 25, 2023 16:42

Add tests to cover ApplyOnce reentry and idempotency

9f84d7e

g-gaston requested a review from fabriziopandini January 26, 2023 15:34

sbueringer reviewed Feb 3, 2023

View reviewed changes

g-gaston and others added 3 commits February 13, 2023 12:21

Fix typos

aed9907

Co-authored-by: Stefan Büringer <[email protected]>

Fix exp/addons/internal/controllers/clusterresourceset_controller_tes…

d4635bb

…t.go typo Co-authored-by: Stefan Büringer <[email protected]>

Improve generate namespace function

c32cf26

g-gaston requested review from sbueringer and removed request for fabriziopandini February 13, 2023 19:37

k8s-ci-robot assigned fabriziopandini and sbueringer Feb 14, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 14, 2023

fabriziopandini reviewed Feb 19, 2023

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 19, 2023

k8s-ci-robot merged commit 5e96566 into kubernetes-sigs:main Feb 19, 2023

k8s-ci-robot added this to the v1.4 milestone Feb 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Implement Reconcile mode for ClusterResourceSet #7497

✨ Implement Reconcile mode for ClusterResourceSet #7497

g-gaston commented Nov 4, 2022 •

edited

Loading

k8s-ci-robot commented Nov 4, 2022

jonathanmeier5 left a comment

g-gaston commented Nov 8, 2022

sbueringer left a comment

sbueringer commented Nov 16, 2022

sbueringer left a comment •

edited

Loading

sbueringer Jan 20, 2023 •

edited

Loading

sbueringer commented Feb 3, 2023

sbueringer left a comment •

edited

Loading

sbueringer Feb 3, 2023

g-gaston Feb 13, 2023

sbueringer Feb 14, 2023 •

edited

Loading

sbueringer commented Feb 14, 2023

k8s-ci-robot commented Feb 14, 2023

fabriziopandini left a comment

k8s-ci-robot commented Feb 19, 2023

		// The create call is idempotent, so if the object already exists
		// then do not consider it to be an error.

	missingNamespace := randomNamespaceForTest(t)
	missingNamespace := fmt.Sprintf("clusterresourceset-apply-once-%s", util.RandomString(6))

✨ Implement Reconcile mode for ClusterResourceSet #7497

✨ Implement Reconcile mode for ClusterResourceSet #7497

Conversation

g-gaston commented Nov 4, 2022 • edited Loading

k8s-ci-robot commented Nov 4, 2022

jonathanmeier5 left a comment

Choose a reason for hiding this comment

g-gaston commented Nov 8, 2022

sbueringer left a comment

Choose a reason for hiding this comment

sbueringer commented Nov 16, 2022

sbueringer left a comment • edited Loading

Choose a reason for hiding this comment

sbueringer Jan 20, 2023 • edited Loading

Choose a reason for hiding this comment

sbueringer commented Feb 3, 2023

sbueringer left a comment • edited Loading

Choose a reason for hiding this comment

sbueringer Feb 3, 2023

Choose a reason for hiding this comment

g-gaston Feb 13, 2023

Choose a reason for hiding this comment

sbueringer Feb 14, 2023 • edited Loading

Choose a reason for hiding this comment

sbueringer commented Feb 14, 2023

k8s-ci-robot commented Feb 14, 2023

fabriziopandini left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Feb 19, 2023

g-gaston commented Nov 4, 2022 •

edited

Loading

sbueringer left a comment •

edited

Loading

sbueringer Jan 20, 2023 •

edited

Loading

sbueringer left a comment •

edited

Loading

sbueringer Feb 14, 2023 •

edited

Loading