Skip to content

Commit

Permalink
sstable: unify sstable size estimation by using a single abstraction
Browse files Browse the repository at this point in the history
The index block size estimation, and the sstable size estimation for data blocks
had similar code, which could be abstracted into a single type. This pr does that.
  • Loading branch information
bananabrick committed Feb 23, 2022
1 parent bac6da8 commit 85162b6
Show file tree
Hide file tree
Showing 3 changed files with 257 additions and 125 deletions.
75 changes: 75 additions & 0 deletions sstable/testdata/size_estimate
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Sequence of ops which tests all of the code paths in the size_estimate type.

init 1
----
success

# Empty size should be 1
size
----
1

# There's a single inflight entry, so the size should be 4.
add_inflight 4
----
4

# After compression, entry only had a size of 3. The total size is the 3, but the
# max estimated size yet is 4.
entry_written 3 4 3
----
4

# Compression ratio is 0.75 at this point. The total size is 3, and the inflight size is
# 4, so that returned size is uint64(6.75).
add_inflight 5
----
6

# We don't clear the empty size, so even after clearing a size of 1 is returned.
clear
----
1

# Test writing multiple inflight entries.
add_inflight 4
----
4

add_inflight 5
----
9

# First inflight entry written. The entry didn't get compressed. The total size now is less than 9,
# but the max estimated size should still be 9.
entry_written 4 4 4
----
9

# At this point, inflightSize is 13, the totalSize is 4. The compression ratio is 1. So, the returned
# size should be 17.
add_inflight 8
----
17

# One entry has been written.
num_entries
----
1

# The inflight entry had a size of 5, but the entry added had a size of 3 because of compression/size estimation.
# The compression ratio is 0.77 at this point. The inflightSize is 8. The true size is 13.16, but the maxEstimatedSize
# is returned.
entry_written 7 5 3
----
17

# The inflight size is 0, and the total size is 11.
entry_written 11 8 4
----
17

# The compression ratio is 0.64, and the inflight size is 20, 20*0.64 = 12.8, so the total size is uint64(12.8 + 11)
add_inflight 20
----
23
Loading

0 comments on commit 85162b6

Please sign in to comment.