Fixes pre-deploy step having incorrect desired labels and adds a temp fix for race condition #835
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gardener-robot
added
area/control-plane
gardener-robot-ci-2
added
the
reviewed/ok-to-test
gardener-robot-ci-1
added
needs/ok-to-test
|
/retest |
1 similar comment
|
/retest |
|
/test pull-etcd-druid-e2e-kind-nondistroless-etcd |
3 similar comments
|
/test pull-etcd-druid-e2e-kind-nondistroless-etcd |
|
/test pull-etcd-druid-e2e-kind-nondistroless-etcd |
|
/test pull-etcd-druid-e2e-kind-nondistroless-etcd |
gardener-robot
added
reviewed/lgtm
gardener-robot-ci-2
added
the
reviewed/ok-to-test
gardener-robot
added
the
status/closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
How to categorize this PR?
/area control-plane
/kind bug
What this PR does / why we need it:
In hotfix branch
PreDeploystep was introduced to ensure that etcd-druid is backward compatible with the changes that are made in master w.r.t pod labels and statefulset label-selector. An issue was discovered when running e2e g/g tests.Desired labels passed to utils.ContainsAllDesiredLabels contains the additional labels which are fetched from
etcd.Spec.Labels. When g/g e2e tests are run then in case we upgrade existing etcd cluster from 1 to 3, g/g will add additional networking labels on the etcd resource. These labels will never be present on the pod in thePreDeploystep. This will result in the reconciliation getting stuck at this stage.Deploystep is never called, so the new labels that are added by g/g on the etcd resource never make it to the pods.Additionally we now add one more safeguard in the custodian controller to skip reconciliation if the reconcile annotation is still added on the etcd resource and retry after 10s. This is done in addition to the already present predicate which also does something similar. However the predicate does not prevent requeue of requests due to error in the custodian which can still interfere with the etcd reconciler.
Which issue(s) this PR fixes:
Fixes #836
Special notes for your reviewer:
Release note: