Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable toggling tagged flavors/images and small fixes #282

Conversation

lmiccini
Copy link
Contributor

@lmiccini lmiccini commented Oct 4, 2024

This commit fixes a couple of issues:

  1. Catch case where all the faulty computes are dumping

    In case all the stale_computes are kdumping we want to make sure
    the check_kdump function returns an empty list, otherwise they may
    be evacuated without waiting the dump to be collected.

  2. Wrap reverse dns lookup in try/except block

    If DNS is broken calling socket.gethostbyaddr() could result in
    an unexpected pod crash.
    Let's wrap the call in try/except with the expectation that a
    reverse dns lookup failure should not prevent evacuation,
    worst case scenario we would not capture the memory dump and
    the user will have a "ERROR Could not perform reverse dns lookup for: X"
    in the logs hinting at resolution not working.

It also introduces two new config options:

TAGGED_IMAGES (default=true)
TAGGED_FLAVORS (default=true)

Setting any of these to false will exclude respectively tagged
images or flavors from being considered when deciding which vm
needs to be evacuated, essentially allowing operators to decide
if they want to use either tagged images or flavors, or both.

@openshift-ci openshift-ci bot requested review from dprince and frenzyfriday October 4, 2024 10:29
This commit fixes a couple of issues:

1. Catch case where all the faulty computes are dumping

   In case all the stale_computes are kdumping we want to make sure
   the check_kdump function returns an empty list, otherwise they may
   be evacuated without waiting the dump to be collected.

2. Wrap reverse dns lookup in try/except block

   If DNS is broken calling socket.gethostbyaddr() could result in
   an unexpected pod crash.
   Let's wrap the call in try/except with the expectation that a
   reverse dns lookup failure should not prevent evacuation,
   worst case scenario we would not capture the memory dump and
   the user will have a "ERROR Could not perform reverse dns lookup for: X"
   in the logs hinting at resolution not working.

It also introduces two new config options:

TAGGED_IMAGES (default=true)
TAGGED_FLAVORS (default=true)

Setting any of these to false will exlude respectively tagged
images or flavors from being considered when deciding which vm
needs to be evacuated, essentially allowing operators to decide
if they want to use either tagged images or flavors, or both.

Co-authored-by: Antonio Romito <[email protected]>
@lmiccini lmiccini force-pushed the kdump-emtpy-computes branch from 6ad2262 to ae42ac0 Compare October 4, 2024 13:17
@lmiccini lmiccini changed the title Catch case where all the faulty computes are dumping Enable toggling tagged flavors/images and small fixes Oct 4, 2024
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/ec59d3fe7b1844e6b1c57ee8f1f032d8

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 11m 38s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 23m 39s
cifmw-crc-podified-edpm-baremetal FAILURE in 30m 10s

@lmiccini
Copy link
Contributor Author

lmiccini commented Oct 5, 2024

recheck

1 similar comment
@danpawlik
Copy link

recheck

@dciabrin
Copy link
Contributor

dciabrin commented Oct 7, 2024

/lgtm

Copy link
Contributor

openshift-ci bot commented Oct 8, 2024

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: lmiccini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit d1d7314 into openstack-k8s-operators:main Oct 8, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants