-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make config reloader file writes atomic #962
Conversation
This addresses an issue found in the Prometheus Operator, which reuses this reloader sidecar, but which then also has a second sidecar which may trigger rule-based reloads while the config sidecar is in the middle of writing out its config (in a non-atomic way): prometheus-operator/prometheus-operator#2501 I didn't add a test for this because it's hard to catch the original problem to begin with, but it has happened. Signed-off-by: Julius Volz <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm from my side! 👌
CI fails with:
@juliusv I suspect you verified that the normal flow of the reloader works with this change? |
I guess errcheck is a bit aggressive here, as the |
Its definitely not aggressive (: it's just a matter of being explicit when you don't need error by doing: |
Signed-off-by: Julius Volz <[email protected]>
I added an explicit error ignore now.
Nope :) Not beyond the existing test suite at least. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thx, LGTM
pkg/reloader/reloader.go
Outdated
@@ -199,9 +199,14 @@ func (r *Reloader) apply(ctx context.Context) error { | |||
return errors.Wrap(err, "expand environment variables") | |||
} | |||
|
|||
if err := ioutil.WriteFile(r.cfgOutputFile, b, 0666); err != nil { | |||
tmpFile := r.cfgOutputFile + ".tmp" | |||
defer os.Remove(tmpFile) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can satisfy errcheck by
defer func(){ _ := os.Remove(tmpFile) }()
This addresses an issue found in the Prometheus Operator, which reuses
this reloader sidecar, but which then also has a second sidecar which
may trigger rule-based reloads while the config sidecar is in the middle
of writing out its config (in a non-atomic way):
prometheus-operator/prometheus-operator#2501
I didn't add a test for this because it's hard to catch the original
problem to begin with, but it has happened.
Signed-off-by: Julius Volz [email protected]