-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
storage: use concrete pebbleIterator
in intentInterleavingIter
#86440
storage: use concrete pebbleIterator
in intentInterleavingIter
#86440
Conversation
ecdccc2
to
ffaa19b
Compare
Release justification: non-production code changes Release note: None
This patch splits up `maybeSkipIntentRangeKeys()` and `maybeSuppressRangeKeyChanged()` to allow for mid-stack inlining. I doubt that the gains are as large as these microbenchmarks claim, and there's a fair bit of variance between runs, but it can't hurt. ``` name old time/op new time/op delta MVCCScan_Pebble/rows=1/versions=1/valueSize=64/numRangeKeys=0-24 5.37µs ± 2% 5.43µs ± 2% +1.13% (p=0.041 n=10+10) MVCCScan_Pebble/rows=100/versions=1/valueSize=64/numRangeKeys=0-24 38.2µs ± 2% 38.2µs ± 2% ~ (p=0.971 n=10+10) MVCCScan_Pebble/rows=10000/versions=1/valueSize=64/numRangeKeys=0-24 2.79ms ± 2% 2.71ms ± 2% -2.59% (p=0.000 n=10+10) MVCCReverseScan_Pebble/rows=1/versions=1/valueSize=64/numRangeKeys=0-24 5.99µs ± 1% 5.99µs ± 2% ~ (p=0.541 n=10+10) MVCCReverseScan_Pebble/rows=100/versions=1/valueSize=64/numRangeKeys=0-24 51.7µs ± 3% 52.1µs ± 1% ~ (p=0.631 n=10+10) MVCCReverseScan_Pebble/rows=10000/versions=1/valueSize=64/numRangeKeys=0-24 3.88ms ± 2% 3.87ms ± 1% ~ (p=0.897 n=10+8) MVCCComputeStats_Pebble/valueSize=32/numRangeKeys=0-24 158ms ± 1% 155ms ± 1% -2.34% (p=0.000 n=10+9) ``` Release justification: bug fixes and low-risk updates to new functionality Release note: None
Since `intentInterleavingIter` always constructs the underlying iterators from the given reader, and these readers always construct `*pebbleIterator`, it can use the concrete type rather than interfaces for both iterators. This avoids dynamic dispatch, yielding a decent performance improvement. Unfortunately, this requires disabling `unsafeMVCCIterator` inside `intentInterleavingIter`. This wasn't always enabled anyway, since it was omitted both for the engine iterator and when using an engine directly (which doesn't have consistent iterator). ``` name old time/op new time/op delta MVCCScan_Pebble/rows=1/versions=1/valueSize=64/numRangeKeys=0-24 5.43µs ± 2% 5.45µs ± 2% ~ (p=0.566 n=10+10) MVCCScan_Pebble/rows=100/versions=1/valueSize=64/numRangeKeys=0-24 38.2µs ± 2% 37.0µs ± 1% -3.02% (p=0.000 n=10+10) MVCCScan_Pebble/rows=10000/versions=1/valueSize=64/numRangeKeys=0-24 2.71ms ± 2% 2.66ms ± 1% -1.83% (p=0.000 n=10+9) MVCCReverseScan_Pebble/rows=1/versions=1/valueSize=64/numRangeKeys=0-24 5.99µs ± 2% 5.86µs ± 2% -2.15% (p=0.000 n=10+10) MVCCReverseScan_Pebble/rows=100/versions=1/valueSize=64/numRangeKeys=0-24 52.1µs ± 1% 50.2µs ± 2% -3.77% (p=0.000 n=10+10) MVCCReverseScan_Pebble/rows=10000/versions=1/valueSize=64/numRangeKeys=0-24 3.87ms ± 1% 3.83ms ± 1% -1.26% (p=0.000 n=8+10) MVCCComputeStats_Pebble/valueSize=32/numRangeKeys=0-24 155ms ± 1% 155ms ± 1% ~ (p=0.842 n=9+10) ``` Release justification: low risk, high benefit changes to existing functionality Release note: None
ffaa19b
to
e209dc1
Compare
CC @nvanbenschoten, this should make up for some of the regression since 22.1. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 4 files at r1, 1 of 1 files at r2.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @sumeerbhola and @tbg)
TFTR! bors r=tbg |
Build succeeded: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 4 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained
-- commits
line 41 at r3:
I think we should try to address this and not take this testing loss. One possibility is to move the simple functionality provided by unsafeMVCCIterator
into pebbleIterator
and do it when RaceEnabled
. This will be a net improvement since engine iterators will also have it.
btw, I did not understand the "omitted ... when using an engine directly" part -- Pebble.NewMVCCIterator
does have the wrapping code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained
-- commits
line 41 at r3:
Wrote up #86657, will have a look during stability.
btw, I did not understand the "omitted ... when using an engine directly" part -- Pebble.NewMVCCIterator does have the wrapping code.
It does, but for readers without ConsistentIterators()
we instead construct the pebbleIterator
by cloning the engine iterator:
cockroach/pkg/storage/intent_interleaving_iter.go
Lines 274 to 275 in e209dc1
iter = newPebbleIteratorByCloning( | |
intentIter.GetRawIter(), opts, StandardDurability, reader.SupportsRangeKeys()) |
storage: tweak
unsafeMVCCIterator
constructionRelease justification: non-production code changes
Release note: None
storage: inline some
intentInterleavingIter
methodsThis patch splits up
maybeSkipIntentRangeKeys()
andmaybeSuppressRangeKeyChanged()
to allow for mid-stack inlining.I doubt that the gains are as large as these microbenchmarks claim, and
there's a fair bit of variance between runs, but it can't hurt.
Touches #82559.
Release justification: bug fixes and low-risk updates to new functionality
Release note: None
storage: use concrete
pebbleIterator
inintentInterleavingIter
Since
intentInterleavingIter
always constructs the underlyingiterators from the given reader, and these readers always construct
*pebbleIterator
, it can use the concrete type rather than interfacesfor both iterators. This avoids dynamic dispatch, yielding a decent
performance improvement.
Unfortunately, this requires disabling
unsafeMVCCIterator
insideintentInterleavingIter
. This wasn't always enabled anyway, since itwas omitted both for the engine iterator and when using an engine
directly (which doesn't have consistent iterators).
Touches #82559.
Release justification: low risk, high benefit changes to existing functionality
Release note: None