Lock bootstrap data with empty key to prevent conflicts #7215
Conversation
Codecov ReportPatch coverage has no change and project coverage change:
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more Additional details and impacted files@@ Coverage Diff @@
## master #7215 +/- ##
=========================================
- Coverage 9.84% 9.83% -0.02%
=========================================
Files 147 147
Lines 10819 10831 +12
=========================================
Hits 1065 1065
- Misses 9532 9544 +12
Partials 222 222
Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
brandond
force-pushed
the
fix_bootstrap_race
branch
2 times, most recently
from
ae574ca to
9748c4f
Compare
| ticker := time.NewTicker(5 * time.Second) | ||
| defer ticker.Stop() | ||
|
|
||
| RETRY: | ||
| for { | ||
| value, c.saveBootstrap, err = getBootstrapKeyFromStorage(ctx, storageClient, normalizedToken, token) | ||
| if err != nil { | ||
| if strings.Contains(err.Error(), "not supported for learner") { | ||
| for range ticker.C { | ||
| continue RETRY | ||
| } | ||
|
|
||
| } | ||
| return err | ||
| } | ||
| if value == nil { | ||
| return nil | ||
| } | ||
|
|
||
| dbRawData, err = decrypt(normalizedToken, value.Data) | ||
| if err != nil { | ||
| return err | ||
| } |
There was a problem hiding this comment.
I moved this whole ugly for/continue loop into a wait.PollImmediateUntilWithContext in the getBootstrapKeyFromStorage function itself.
brandond
force-pushed
the
fix_bootstrap_race
branch
from
9748c4f to
81f3968
Compare
Signed-off-by: Brad Davidson <[email protected]>
brandond
force-pushed
the
fix_bootstrap_race
branch
from
81f3968 to
6f84acd
Compare
Proposed Changes
Lock bootstrap data with empty key to prevent conflicts
Types of Changes
Bugfix
Verification
This has been hard to reproduce on demand, as the timing involved is very tight. Basic test would be to launch a multi-server cluster using an external SQL DB where all the servers are started as close to simultaneously as possible.
Testing
Linked Issues
User-Facing Change
Further Comments