Move singletons out of Faster instance #30
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I am observing issues with uploading large files: CloudBlockBlob.UploadFromStreamAsync: Microsoft.Azure.Storage.StorageException: The specified blob or block content is invalid. I think it is caused by a partial upload that fails, and then the next upload uses different blocksizes which just does not work. I may need to use unique names for these blobs. |
sebastianburckhardt
force-pushed
the
pr/separate-singleton-storage
branch
from
4ffb1f0 to
2e118bc
Compare
|
I changed the design so that singletons are stored in a separate file inside the cpr-checkpoint folder. This means I do not have to worry about leases, and partial uploads cannot lead to permanently stuck states. |
sebastianburckhardt
force-pushed
the
pr/separate-singleton-storage
branch
from
2ab598e to
1eb6c06
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Singleton objects clog Faster's object log. Get better perf by storing instance objects separately, and snapshotting them synchronously when creating checkpoing. We store them inside checkpoint commit file.