Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(deployment): Clean up orphaned deployments #759

Merged
merged 3 commits into from
Apr 13, 2019
Merged

fix(deployment): Clean up orphaned deployments #759

merged 3 commits into from
Apr 13, 2019

Conversation

dinhxuanvu
Copy link
Member

Signed-off-by: Vu Dinh [email protected]

@openshift-ci-robot openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 14, 2019
@dinhxuanvu dinhxuanvu requested review from ecordell and removed request for jpeeler and alecmerdler March 14, 2019 16:41
@dinhxuanvu
Copy link
Member Author

/test e2e-aws-olm

ecordell
ecordell reviewed Mar 15, 2019
View reviewed changes
Copy link
Member

@ecordell ecordell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great start! See comments inline - this is a good opportunity to improve our deployment strategy installer, so I made some suggestions around moving this there.

pkg/controller/operators/olm/operator.go Outdated
for _, d := range existingDeployments {
if _, exists := depNames[d.GetName()]; !exists {
logger.Infof("found an orphaned deployment %s in namespace %s", d.GetName(), csv.GetNamespace())
if err := a.OpClient.DeleteDeployment(csv.GetNamespace(), d.GetName(), &metav1.DeleteOptions{}); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before deleting anything, we should probably check the ownerreferences to be extra safe. it there is an orphaned deployment, it should have an ownerreference to this CSV

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The owner check has already been done in line #1399 above:

https://github.com/operator-framework/operator-lifecycle-manager/pull/759/files#diff-c09af0ae4ca3b7464670dbd80be66c70R1399

The existingDeployments should only contain deployments that belong to to a specific CSV out based on the labels.selector filter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That does query by the labels which we use for finding the objects, but for these objects (since they're in the same namespace) we add "real" owner references as well. It's better to be extra safe when GCing

pkg/controller/operators/olm/operator.go Outdated Show resolved Hide resolved
@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 15, 2019
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 15, 2019
@dinhxuanvu dinhxuanvu force-pushed the deployment-update branch 4 times, most recently from 15f4c15 to c824d05 Compare March 19, 2019 15:25
ecordell
ecordell reviewed Mar 19, 2019
View reviewed changes
Copy link
Member

@ecordell ecordell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's looking great! I think we can clean up some older stuff now that this is here, so I made some comments about that.

pkg/controller/install/deployment.go Show resolved Hide resolved
pkg/controller/operators/olm/operator.go Outdated
@@ -906,6 +906,10 @@ func (a *Operator) transitionCSVState(in v1alpha1.ClusterServiceVersion) (out *v
}
}

// Clean up deployments in case some become orphaned
if cleanupErr := a.cleanupOrphanedDeployments(logger, out); cleanupErr == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part can be removed now right?

pkg/api/wrappers/deployment_install_client.go Show resolved Hide resolved
pkg/api/wrappers/deployment_install_client.go Outdated
@@ -118,3 +120,13 @@ func (c *InstallStrategyDeploymentClientForNamespace) FindAnyDeploymentsMatching
}
return deployments, nil
}

func (c *InstallStrategyDeploymentClientForNamespace) FindAnyDeploymentsMatchingLabelSelector(label labels.Selector) ([]*appsv1.Deployment, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a unit test for this?

pkg/controller/install/deployment.go Show resolved Hide resolved
@ecordell ecordell added the kind/bug Categorizes issue or PR as related to a bug. label Mar 19, 2019
@dinhxuanvu dinhxuanvu force-pushed the deployment-update branch 2 times, most recently from 9ee716a to bd3aab2 Compare March 19, 2019 20:09
@openshift-ci-robot openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 19, 2019
@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 27, 2019
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 27, 2019
@dinhxuanvu dinhxuanvu force-pushed the deployment-update branch 3 times, most recently from ca3f528 to 56ac5cf Compare March 27, 2019 06:57
@dinhxuanvu
Copy link
Member Author

/test e2e-aws

@openshift-ci-robot openshift-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 11, 2019
@dinhxuanvu dinhxuanvu force-pushed the deployment-update branch 4 times, most recently from c05a32d to f4e03f3 Compare April 12, 2019 12:51
@ecordell
Copy link
Member

Looks great, thanks for wrapping this up!

/lgtm
/retest

@openshift-ci-robot openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Apr 12, 2019
@dinhxuanvu
Copy link
Member Author

/retest

2 similar comments
@dinhxuanvu
Copy link
Member Author

/retest

@dinhxuanvu
Copy link
Member Author

/retest

@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 13, 2019
@openshift-ci-robot openshift-ci-robot removed lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 13, 2019
@dinhxuanvu dinhxuanvu force-pushed the deployment-update branch 2 times, most recently from 2590afa to f4e03f3 Compare April 13, 2019 06:19
@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 13, 2019
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 13, 2019
@ecordell
Copy link
Member

/retest

@ecordell
Copy link
Member

/lgtm
/retest

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 13, 2019
@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dinhxuanvu, ecordell

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 4de8dd1 into operator-framework:master Apr 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@njhale njhale njhale left review comments

@ecordell ecordell ecordell left review comments

Assignees

@ecordell ecordell

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@dinhxuanvu @ecordell @openshift-ci-robot @njhale @openshift-merge-robot