database plugin: Invalidate queue should cancel context first #15933
Conversation
To signal to any credentials rotating goroutines that they should cancel pending operations, which reduces lock contention.
| if b.cancelQueue != nil { | ||
| b.cancelQueue() | ||
| } | ||
| b.Lock() |
There was a problem hiding this comment.
Technically I would argue we don't need the lock, and we don't need to nil out credRotationQueue, since that field is only initialized by Factory, so it's not like databaseBackend can get usefully recycled. But it's probably safest to make the minimal change, so ignore me.
There was a problem hiding this comment.
+1, doesn't seem like the lock is needed. I'm also okay with leaving it.
There was a problem hiding this comment.
That's true, though there are several places where we check whether credRotationQueue nil or not and only perform things like rotation (async) if non-nil so not sure if it'd impact those calls if we were to skip setting it to nil here.
There was a problem hiding this comment.
Like @calvn says, I think we have to leave the lock since there are several times when we check if credRotationQueue is nil and if not, then read from it. Not having the lock would introduce a potential (unlikely) race condition where we set credRotationQueue nil here, but after the != nil check elsewhere but before they use credRotationQueue.
There was a problem hiding this comment.
The more I think about it, the more I think we can safely use a different pattern instead of doing the credRotationQueue != nil pattern. Something like:
select {
case <-b.ctx.Done():
default:
// credRotationQueue is valid, use it
}And instead of setting credRotationQueue = nil, we just rely on the context that we're canceling anyway.
I'll see if I can take a stab at this, and limiting the use of the other lock, in another PR I think.
|
Thanks everyone! |
Following up from discussions in #15923 and #15933, I wanted to split out a separate PR that drastically reduced the complexity of the use of the databaseBackend lock. We no longer need it at all for the `credRotationQueue`, and we can move it to be solely used in a few, small connections map management functions.
Cleanup and simplify lock usage in database plugin Following up from discussions in #15923 and #15933, I wanted to split out a separate PR that drastically reduced the complexity of the use of the databaseBackend lock. We no longer need it at all for the `credRotationQueue`, and we can move it to be solely used in a few, small connections map management functions. Co-authored-by: Calvin Leung Huang <[email protected]>
To signal to any credentials rotating goroutines that they should cancel
pending operations, which reduces lock contention.