Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1877281: [vSphere] Don't fail when tag is not found #667

Merged

Conversation

alexander-demicev
Copy link
Contributor

When a cluster that was first created via UPI the clusterID tag is missed in the vCenter.
This causes the machine controller to create the instances but failing and breaking the loop before being able to reconcile the status appropriately resulting in the machine resource having an inconsistent status which does not reflect reality.

github.com/vmware/govmomi misses typed errors, so the only way to not fail reconciliation on not found tag is iterate through the list of all tags and try to find clusterID tag. We assume the tag is always present in IPI.

@JoelSpeed
Copy link
Contributor

Is there a BZ related to this?

Can you please ensure that there are unit tests to cover this at a minimum

@alexander-demicev
Copy link
Contributor Author

We don't have a BZ, it's a jira ticket

Copy link
Contributor

@michaelgugino michaelgugino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know the background of this ticket, but the UPI cluster needs to conform to some standard. It seems like they should supply the correct tagging on whatever it is they're doing rather than us not utilizing it. Cluster tags are important if you're running more than one cluster on the same set of infrastructure.

@alexander-demicev
Copy link
Contributor Author

alexander-demicev commented Aug 12, 2020

@michaelgugino What you say is true, do we require cluster tags for other providers' UPI? Do you know if that's documented somewhere?

@michaelgugino
Copy link
Contributor

michaelgugino commented Aug 13, 2020

@michaelgugino What you say is true, do we require cluster tags for other providers' UPI? Do you know if that's documented somewhere?

I skimmed through the vSphere UPI docs, I didn't see anything about tags.

I am not sure how everything works in this provider. If we can able to query machines from vsphere on a unique basis (eg, every VM has a unique name that we know), we might be able to ignore the missing tags for clusterid.

@@ -882,6 +882,16 @@ func (vm *virtualMachine) reconcileTags(ctx context.Context, session *session.Se

// checkAttachedTag returns true if tag is already attached to a vm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the comment needs to be updated

@enxebre
Copy link
Member

enxebre commented Sep 9, 2020

In vSphere we use the s.machine.UID or fallback to the s.machine.Name to find instances. As far as I'm aware the clusterID tag is only used by the installer to remove resources on a destroy command so this change seems safe. This would let UPI born clusters to work smoothly.

I also agree with this though #667 (review)
https://bugzilla.redhat.com/show_bug.cgi?id=1877281

@enxebre
Copy link
Member

enxebre commented Sep 9, 2020

/retest

@enxebre
Copy link
Member

enxebre commented Sep 10, 2020

/retitle Bug 1877281: [vSphere] Don't fail when tag is not found

@openshift-ci-robot openshift-ci-robot changed the title [vSphere] Don't fail when tag is not found Bug 1877281: [vSphere] Don't fail when tag is not found Sep 10, 2020
@openshift-ci-robot openshift-ci-robot added bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Sep 10, 2020
@openshift-ci-robot
Copy link
Contributor

@alexander-demichev: This pull request references Bugzilla bug 1877281, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.6.0) matches configured target release for branch (4.6.0)
  • bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1877281: [vSphere] Don't fail when tag is not found

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@enxebre
Copy link
Member

enxebre commented Sep 10, 2020

/approve

@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: enxebre

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 10, 2020
Copy link
Contributor

@JoelSpeed JoelSpeed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 14, 2020
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Sep 14, 2020

@alexander-demichev: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-aws-scaleup-rhel7 5dceceb link /test e2e-aws-scaleup-rhel7
ci/prow/e2e-azure-operator 5dceceb link /test e2e-azure-operator
ci/prow/e2e-aws-workers-rhel7 6fc674c link /test e2e-aws-workers-rhel7

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit a682acd into openshift:master Sep 14, 2020
@openshift-ci-robot
Copy link
Contributor

@alexander-demichev: All pull requests linked via external trackers have merged:

Bugzilla bug 1877281 has been moved to the MODIFIED state.

In response to this:

Bug 1877281: [vSphere] Don't fail when tag is not found

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants