promtail: write positions to new file first, move to target location afterwards #1304
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rds. Signed-off-by: Peter Štibraný <[email protected]>
|
If the rename fails, should the new file be deleted? WDYT? |
Nah, it will be overwritten on next try, or can be left there for inspection / manual rename by admin. |
|
Should we use https://github.com/natefinch/atomic/ ? |
From what I can tell, it's doing exactly the same as Go stdlib: calls rename on Unix/Linux, and uses same MoveFileEx system call on Windows with MOVEFILE_REPLACE_EXISTING flag that Go stdlib does. Perhaps Go didn't do that in the past? This lib seems to be 5+ years old. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
promtail uses positions file to write positions in watched log files. It's doing so in an unsafe way, overwriting existing file. If error happens while writing to the file, file ends up being inconsistent. And old version is gone.
This PR changes positions writing, so that it's first written to new file, and only if that succeeds, is it move to target position. New file is file in the same directory, but with "-new" appended.
os.Rename guarantees that it replaces existing file, or it fails, but it cannot leave file inconsistent.