Skip to content

Commit

Permalink
nfd-master: stop node-updater pool before reconfiguring api-controller
Browse files Browse the repository at this point in the history
Prevents potential race between node-updater pool and the api-controller
when re-configuring nfd-master. Reconfiguration causes a new
api-controller instance to be created so nfd api lister might change in
the midst of processing a node update (if the pool was running). No
actual issues related to this have been identified but races (like this)
should still be avoided.
  • Loading branch information
marquiz committed Apr 9, 2024
1 parent 31a56ac commit ba4cebb
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions pkg/nfd-master/nfd-master.go
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,11 @@ func (m *nfdMaster) Run() error {
return err
}

// Stop the nodeUpdaterPool so that no node updates are underway
// while we reconfigure the NFD API controller (including the

Check warning on line 354 in pkg/nfd-master/nfd-master.go

View check run for this annotation

Codecov / codecov/patch

pkg/nfd-master/nfd-master.go#L353-L354

Added lines #L353 - L354 were not covered by tests
// listers) below
m.nodeUpdaterPool.stop()

// restart NFD API controller
if m.nfdController != nil {
klog.InfoS("stopping the nfd api controller")
Expand All @@ -361,13 +366,13 @@ func (m *nfdMaster) Run() error {
return nil
}
}
// Restart the nodeUpdaterPool

Check warning on line 369 in pkg/nfd-master/nfd-master.go

View check run for this annotation

Codecov / codecov/patch

pkg/nfd-master/nfd-master.go#L369

Added line #L369 was not covered by tests
m.nodeUpdaterPool.start(m.config.NfdApiParallelism)

// Update all nodes when the configuration changes
if m.nfdController != nil && features.NFDFeatureGate.Enabled(features.NodeFeatureAPI) {
m.nfdController.updateAllNodesChan <- struct{}{}
}
// Restart the node updater pool
m.nodeUpdaterPool.stop()
m.nodeUpdaterPool.start(m.config.NfdApiParallelism)

case <-m.stop:
klog.InfoS("shutting down nfd-master")
Expand Down

0 comments on commit ba4cebb

Please sign in to comment.