Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport of drainer: test refactoring to clarify behavior around delete/down nodes into release/1.5.x #16622

Conversation

hc-github-team-nomad-core
Copy link
Contributor

Backport

This PR is auto-generated from #16612 to be assessed for backporting due to the inclusion of the label backport/1.5.x.

The below text is copied from the body of the original PR.


This changeset refactors the tests of the draining node watcher so that we don't
mock the node watcher's Remove and Update methods for its own tests. Instead
we'll mock the node watcher's dependencies (the job watcher and deadline
notifier) and now unit tests can cover the real code. This allows us to remove a
bunch of TODOs in watch_nodes.go around testing and clarify some important
behaviors:

  • Nodes that are down or disconnected will still be watched until the scheduler
    decides what to do with their allocations. This will drive the job watcher but
    not the node watcher, and that lets the node watcher gracefully handle cases
    where a heartbeat fails but the node heartbeats again before its allocs can be
    evicted.

  • Stop watching nodes that have been deleted. The blocking query for nodes set
    the maximum index to the highest index of a node it found, rather than the
    index of the nodes table. This misses updates to the index from deleting
    nodes. This was done as an performance optimization to avoid excessive
    unblocking, but because the query is over all nodes anyways there's no
    optimization to be had here. Remove the optimization so we can detect deleted
    nodes without having to wait for an update to an unrelated node.

@hc-github-team-nomad-core hc-github-team-nomad-core merged commit e865333 into release/1.5.x Mar 23, 2023
@hc-github-team-nomad-core hc-github-team-nomad-core force-pushed the backport/drain-legibility-testing/ultimately-legal-macaw branch from 94d1b77 to de741fe Compare March 23, 2023 18:07
@hc-github-team-nomad-core hc-github-team-nomad-core deleted the backport/drain-legibility-testing/ultimately-legal-macaw branch March 23, 2023 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants