Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: optimize MVCCExportToSST range key processing #86521

Merged
merged 1 commit into from
Aug 22, 2022

Conversation

erikgrinaker
Copy link
Contributor

@erikgrinaker erikgrinaker commented Aug 21, 2022

This patch optimizes MVCCExportToSST by making use of
RangeKeyChanged(), CloneInto(), and PutRawMVCCRangeKey().
Unfortunately, this did not appear to have a significant impact on
performance in the 0/1 range key cases, as most of the gains were
already realized by previously migrating MVCCIncrementalIterator to
RangeKeyChanged().

name                                                                                      old time/op  new time/op   delta
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24      69.8µs ± 0%   69.9µs ± 1%    ~     (p=0.841 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24       116µs ± 7%    113µs ± 6%    ~     (p=0.548 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24       66.9µs ± 1%   66.6µs ± 1%    ~     (p=0.151 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24       98.4µs ± 7%  105.6µs ± 4%    ~     (p=0.095 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24      171µs ± 4%    166µs ± 4%    ~     (p=0.222 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24      229µs ± 4%    230µs ± 3%    ~     (p=0.841 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24       277µs ± 3%    270µs ± 2%    ~     (p=0.151 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24       345µs ± 2%    346µs ± 1%    ~     (p=1.000 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24     617µs ± 1%    614µs ± 1%    ~     (p=0.310 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24     762µs ± 1%    756µs ± 1%    ~     (p=0.151 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24      571µs ± 2%    556µs ± 1%  -2.60%  (p=0.032 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24      705µs ± 1%    700µs ± 1%    ~     (p=0.222 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24   2.17ms ± 0%   2.19ms ± 1%  +0.80%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24   2.87ms ± 1%   2.91ms ± 1%  +1.53%  (p=0.032 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24    3.80ms ± 0%   3.79ms ± 1%    ~     (p=0.548 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24    4.77ms ± 0%   4.75ms ± 0%    ~     (p=0.095 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24    4.63ms ± 1%   4.57ms ± 1%  -1.24%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24    5.61ms ± 0%   5.65ms ± 1%  +0.66%  (p=0.032 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24     4.21ms ± 1%   4.14ms ± 0%  -1.69%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24     5.23ms ± 0%   5.17ms ± 1%  -1.16%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24   17.6ms ± 1%   17.8ms ± 2%    ~     (p=0.222 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24   23.3ms ± 0%   23.6ms ± 0%  +1.10%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24    30.9ms ± 0%   30.7ms ± 0%  -0.51%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24    38.5ms ± 0%   38.4ms ± 0%    ~     (p=0.310 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24   36.6ms ± 0%   36.1ms ± 1%  -1.46%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24   44.5ms ± 0%   44.6ms ± 0%    ~     (p=0.310 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24    33.4ms ± 1%   32.8ms ± 0%  -1.70%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24    41.1ms ± 1%   40.7ms ± 0%  -1.04%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24   152ms ± 1%    152ms ± 4%    ~     (p=1.000 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24   197ms ± 0%    200ms ± 0%  +1.61%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24    255ms ± 1%    253ms ± 0%  -0.56%  (p=0.032 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24    314ms ± 0%    314ms ± 0%    ~     (p=0.730 n=4+5)

Release justification: bug fixes and low-risk updates to new functionality

Release note: None

@erikgrinaker erikgrinaker requested review from jbowens, tbg and a team August 21, 2022 11:18
@erikgrinaker erikgrinaker self-assigned this Aug 21, 2022
@erikgrinaker erikgrinaker requested review from a team as code owners August 21, 2022 11:18
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@erikgrinaker
Copy link
Contributor Author

I ran a few CPU profiles here to find out why a single range key adds ~25% overhead. It seemed to all be down in Pebble, specifically interleavingIter.

For 23.1, we could reconsider whether we need interleaved exports, and instead export point/range keys separately (which would eliminate any penalty). However, we'll likely have to improve the interleavingIter performance for e.g. the scan case anyway, so it may be moot.

This patch optimizes `MVCCExportToSST` by making use of
`RangeKeyChanged()`, `CloneInto()`, and `PutRawMVCCRangeKey()`.
Unfortunately, this did not appear to have a significant impact on
performance in the 0/1 range key cases, as most of the gains were
already realized by previously migrating `MVCCIncrementalIterator` to
`RangeKeyChanged()`.

```
name                                                                                      old time/op  new time/op   delta
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24      69.8µs ± 0%   69.9µs ± 1%    ~     (p=0.841 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24       116µs ± 7%    113µs ± 6%    ~     (p=0.548 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24       66.9µs ± 1%   66.6µs ± 1%    ~     (p=0.151 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24       98.4µs ± 7%  105.6µs ± 4%    ~     (p=0.095 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24      171µs ± 4%    166µs ± 4%    ~     (p=0.222 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24      229µs ± 4%    230µs ± 3%    ~     (p=0.841 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24       277µs ± 3%    270µs ± 2%    ~     (p=0.151 n=5+5)
MVCCExportToSST/numKeys=64/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24       345µs ± 2%    346µs ± 1%    ~     (p=1.000 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24     617µs ± 1%    614µs ± 1%    ~     (p=0.310 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24     762µs ± 1%    756µs ± 1%    ~     (p=0.151 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24      571µs ± 2%    556µs ± 1%  -2.60%  (p=0.032 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24      705µs ± 1%    700µs ± 1%    ~     (p=0.222 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24   2.17ms ± 0%   2.19ms ± 1%  +0.80%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24   2.87ms ± 1%   2.91ms ± 1%  +1.53%  (p=0.032 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24    3.80ms ± 0%   3.79ms ± 1%    ~     (p=0.548 n=5+5)
MVCCExportToSST/numKeys=1024/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24    4.77ms ± 0%   4.75ms ± 0%    ~     (p=0.095 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24    4.63ms ± 1%   4.57ms ± 1%  -1.24%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24    5.61ms ± 0%   5.65ms ± 1%  +0.66%  (p=0.032 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24     4.21ms ± 1%   4.14ms ± 0%  -1.69%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24     5.23ms ± 0%   5.17ms ± 1%  -1.16%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24   17.6ms ± 1%   17.8ms ± 2%    ~     (p=0.222 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24   23.3ms ± 0%   23.6ms ± 0%  +1.10%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24    30.9ms ± 0%   30.7ms ± 0%  -0.51%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=8192/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24    38.5ms ± 0%   38.4ms ± 0%    ~     (p=0.310 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=false/numRangeKeys=0-24   36.6ms ± 0%   36.1ms ± 1%  -1.46%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=false/numRangeKeys=1-24   44.5ms ± 0%   44.6ms ± 0%    ~     (p=0.310 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=true/numRangeKeys=0-24    33.4ms ± 1%   32.8ms ± 0%  -1.70%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=1/exportAllRevisions=true/numRangeKeys=1-24    41.1ms ± 1%   40.7ms ± 0%  -1.04%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=false/numRangeKeys=0-24   152ms ± 1%    152ms ± 4%    ~     (p=1.000 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=false/numRangeKeys=1-24   197ms ± 0%    200ms ± 0%  +1.61%  (p=0.008 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=true/numRangeKeys=0-24    255ms ± 1%    253ms ± 0%  -0.56%  (p=0.032 n=5+5)
MVCCExportToSST/numKeys=65536/numRevisions=10/exportAllRevisions=true/numRangeKeys=1-24    314ms ± 0%    314ms ± 0%    ~     (p=0.730 n=4+5)
```

Release justification: bug fixes and low-risk updates to new functionality

Release note: None
@erikgrinaker erikgrinaker force-pushed the mvcc-export-rangekeychanged branch from 1ed8200 to d27f8d6 Compare August 22, 2022 16:10
@erikgrinaker
Copy link
Contributor Author

bors r=tbg

@craig
Copy link
Contributor

craig bot commented Aug 22, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Aug 22, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Aug 22, 2022

Build succeeded:

@craig craig bot merged commit 76f5b3a into cockroachdb:master Aug 22, 2022
@erikgrinaker erikgrinaker deleted the mvcc-export-rangekeychanged branch August 23, 2022 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants