Reimplement GC in move-rocks by added CF index #902
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Resolve the problem of GC not being implemented for the key format that ended up being merged, by introducing a time-sorted index CF named "transaction_timeline". The entries are added with each transaction, but are only collected by GC passes when their time window passes.
mzabaluev
changed the title
mzabaluev
requested review from
l-monninger,
0xmovses and
musitdev
as code owners
l-monninger
requested changes
Dec 3, 2024
| while let Some(res) = iter.next() { | ||
| let (timeline_key, key) = res?; | ||
|
|
||
| batch.delete_cf(&timeline_cf_handle, &timeline_key); |
There was a problem hiding this comment.
Two things w.r.t. to this implementation and testing that would be nice to haves:
- I think we might want to fuzz a fair few of these tests--particularly the GC.
- I think we would benefit from using domain-specific types for GC timestamps. I was looking at the usage here and in the other garbage collected containers we have, and it's quite easy to mix up the timestamps with something else.
There was a problem hiding this comment.
Sounds reasonable. As we need custom formatting here, perhaps a newtype defined in the library and implementing Display to format into {:032} would be appropriate.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #879 #880 #881
Summary
protocol-units.Resolve the problem of GC not being implemented for the key format that ended up being merged, by introducing a time-sorted index CF named "transaction_timeline". The entries are added with each transaction, but are only collected by GC passes when their time window passes.
Changelog
Testing
test_rocksdb_gcpasses in the move-rocks crate.