-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
storage: optimize MVCCExportToSST
range key processing
#86521
storage: optimize MVCCExportToSST
range key processing
#86521
Conversation
I ran a few CPU profiles here to find out why a single range key adds ~25% overhead. It seemed to all be down in Pebble, specifically For 23.1, we could reconsider whether we need interleaved exports, and instead export point/range keys separately (which would eliminate any penalty). However, we'll likely have to improve the |
This patch optimizes `MVCCExportToSST` by making use of `RangeKeyChanged()`, `CloneInto()`, and `PutRawMVCCRangeKey()`. Unfortunately, this did not appear to have a significant impact on performance in the 0/1 range key cases, as most of the gains were already realized by previously migrating `MVCCIncrementalIterator` to `RangeKeyChanged()`. ``` name old time/op new time/op delta MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24 69.8µs ± 0% 69.9µs ± 1% ~ (p=0.841 n=5+5) MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24 116µs ± 7% 113µs ± 6% ~ (p=0.548 n=5+5) MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24 66.9µs ± 1% 66.6µs ± 1% ~ (p=0.151 n=5+5) MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24 98.4µs ± 7% 105.6µs ± 4% ~ (p=0.095 n=5+5) MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24 171µs ± 4% 166µs ± 4% ~ (p=0.222 n=5+5) MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24 229µs ± 4% 230µs ± 3% ~ (p=0.841 n=5+5) MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24 277µs ± 3% 270µs ± 2% ~ (p=0.151 n=5+5) MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24 345µs ± 2% 346µs ± 1% ~ (p=1.000 n=5+5) MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24 617µs ± 1% 614µs ± 1% ~ (p=0.310 n=5+5) MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24 762µs ± 1% 756µs ± 1% ~ (p=0.151 n=5+5) MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24 571µs ± 2% 556µs ± 1% -2.60% (p=0.032 n=5+5) MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24 705µs ± 1% 700µs ± 1% ~ (p=0.222 n=5+5) MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24 2.17ms ± 0% 2.19ms ± 1% +0.80% (p=0.008 n=5+5) MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24 2.87ms ± 1% 2.91ms ± 1% +1.53% (p=0.032 n=5+5) MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24 3.80ms ± 0% 3.79ms ± 1% ~ (p=0.548 n=5+5) MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24 4.77ms ± 0% 4.75ms ± 0% ~ (p=0.095 n=5+5) MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24 4.63ms ± 1% 4.57ms ± 1% -1.24% (p=0.008 n=5+5) MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24 5.61ms ± 0% 5.65ms ± 1% +0.66% (p=0.032 n=5+5) MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24 4.21ms ± 1% 4.14ms ± 0% -1.69% (p=0.008 n=5+5) MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24 5.23ms ± 0% 5.17ms ± 1% -1.16% (p=0.008 n=5+5) MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24 17.6ms ± 1% 17.8ms ± 2% ~ (p=0.222 n=5+5) MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24 23.3ms ± 0% 23.6ms ± 0% +1.10% (p=0.008 n=5+5) MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24 30.9ms ± 0% 30.7ms ± 0% -0.51% (p=0.008 n=5+5) MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24 38.5ms ± 0% 38.4ms ± 0% ~ (p=0.310 n=5+5) MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24 36.6ms ± 0% 36.1ms ± 1% -1.46% (p=0.008 n=5+5) MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24 44.5ms ± 0% 44.6ms ± 0% ~ (p=0.310 n=5+5) MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24 33.4ms ± 1% 32.8ms ± 0% -1.70% (p=0.008 n=5+5) MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24 41.1ms ± 1% 40.7ms ± 0% -1.04% (p=0.008 n=5+5) MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24 152ms ± 1% 152ms ± 4% ~ (p=1.000 n=5+5) MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24 197ms ± 0% 200ms ± 0% +1.61% (p=0.008 n=5+5) MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24 255ms ± 1% 253ms ± 0% -0.56% (p=0.032 n=5+5) MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24 314ms ± 0% 314ms ± 0% ~ (p=0.730 n=4+5) ``` Release justification: bug fixes and low-risk updates to new functionality Release note: None
1ed8200
to
d27f8d6
Compare
bors r=tbg |
Build failed (retrying...): |
Build failed (retrying...): |
Build succeeded: |
This patch optimizes
MVCCExportToSST
by making use ofRangeKeyChanged()
,CloneInto()
, andPutRawMVCCRangeKey()
.Unfortunately, this did not appear to have a significant impact on
performance in the 0/1 range key cases, as most of the gains were
already realized by previously migrating
MVCCIncrementalIterator
toRangeKeyChanged()
.Release justification: bug fixes and low-risk updates to new functionality
Release note: None