-
Notifications
You must be signed in to change notification settings - Fork 466
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
db: add range key table stats #1616
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 1 files at r1, 10 of 11 files at r2, all commit messages.
Reviewable status: 10 of 11 files reviewed, 4 unresolved discussions (waiting on @itsbilal and @nicktrav)
-- commits, line 5 at r1:
s/range key/range deletion/
-- commits, line 7 at r1:
Just a heads up, for producing delete-only compactions we'll need to examine both range deletions and range keys in tandem, which might throw a wrench in factoring range keys into their own method.
table_stats.go, line 294 at r1 (raw file):
} // loadTablePointKeyStats calculates the range deletion statistics for the given
s/loadTablePointKeyStats/sloadTableRangeDelStats/
testdata/table_stats, line 184 at r2 (raw file):
Previously, nicktrav (Nick Travers) wrote…
Curiously, these crept in due to the use of the
testkeys.Comparer
. I need to dig into why exactly, but it seems benign.
I bet it's just the comparer name which gets serialized into the sstable properties. The range-deletion-bytes-estimate includes the full size of sstables that are wholly contained by a rangedel, so that does include the properties block.
To improve readability, factor out the logic to populate the point and range deletion table stats into their own separate methods. This also allows for adding a separate method for range keys in the future, if the logic becomes complex enough.
Range key-related statistics are not currently calculated and stored in a table's `FileMetadata`, and are required for compaction picking. Add a `NumRangeKeys` field to `manifest.TableStats` which is populated by the total number of range keys present in a table (the sum of range key sets, unsets and deletes).
f342710
to
a2e377b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TFTR!
Reviewable status: 0 of 11 files reviewed, 3 unresolved discussions (waiting on @itsbilal and @jbowens)
Previously, jbowens (Jackson Owens) wrote…
s/range key/range deletion/
Done.
Previously, jbowens (Jackson Owens) wrote…
Just a heads up, for producing delete-only compactions we'll need to examine both range deletions and range keys in tandem, which might throw a wrench in factoring range keys into their own method.
Ah - good to know. Will see how this plays out.
table_stats.go, line 294 at r1 (raw file):
Previously, jbowens (Jackson Owens) wrote…
s/loadTablePointKeyStats/sloadTableRangeDelStats/
Done.
testdata/table_stats, line 184 at r2 (raw file):
Previously, jbowens (Jackson Owens) wrote…
I bet it's just the comparer name which gets serialized into the sstable properties. The range-deletion-bytes-estimate includes the full size of sstables that are wholly contained by a rangedel, so that does include the properties block.
Ah nice.
func TestLen(t *testing.T) {
fmt.Println(len(testkeys.Comparer.Name)) // 24
fmt.Println(len(base.DefaultComparer.Name)) // 26
}
Two commits in this series - the first is a code shuffle (no logic changes); the second has the additions.
db: factor out some table stats helpers
To improve readability, factor out the logic to populate the point and
range key table stats into their own separate methods. This also allows
for adding a separate method for range keys in the future, if the logic
becomes complex enough.
db: add range key count to table stats
Range key-related statistics are not currently calculated and stored in
a table's
FileMetadata
, and are required for compaction picking.Add a
NumRangeKeys
field tomanifest.TableStats
which is populatedby the total number of range keys present in a table (the sum of range
key sets, unsets and deletes).