rowexec: allow ordered joinReader to stream matches to the first row #85731
Conversation
|
This change is |
There was a problem hiding this comment.
Nice idea!
Reviewed 2 of 2 files at r1, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @DrewKimball)
pkg/sql/rowexec/joinreader_strategies.go line 516 at r1 (raw file):
// on. This could significantly decrease the overhead of buffering looked // up rows. unbufferedRow rowenc.EncDatumRow
nit: "unbuffered" reads unusual to me, maybe "notBuffered" would be better?
pkg/sql/rowexec/joinreader_strategies.go line 654 at r1 (raw file):
if !s.isPartialJoin { if inputRowIdx == 0 { // Don't add to inputRowIdxToLookedUpRowIndices in order to avoid
nit: it'd be good to update the comment on inputRowIdxToLookedUpRowIndices accordingly.
DrewKimball
force-pushed
the
streaming-lookup
branch
from
fe14b62 to
e8c846c
Compare
There was a problem hiding this comment.
Reviewable status:
pkg/sql/rowexec/joinreader_strategies.go line 516 at r1 (raw file):
Previously, yuzefovich (Yahor Yuzefovich) wrote…
nit: "unbuffered" reads unusual to me, maybe "notBuffered" would be better?
Done.
pkg/sql/rowexec/joinreader_strategies.go line 654 at r1 (raw file):
Previously, yuzefovich (Yahor Yuzefovich) wrote…
nit: it'd be good to update the comment on
inputRowIdxToLookedUpRowIndicesaccordingly.
Done.
Currently the `joinReaderOrderingStrategy` implementation buffers all looked up rows before matching them with input rows and emitting them. This is necessary because the looked up rows may not be received in input order (which must be maintained). However, rows that match the first input row can be emitted immediately. In the case when there are many rows that match the first input row, this can decrease overhead of the buffer. Additionally, this change can allow a limit to be satisfied earlier, which can significantly decrease latency. This is especially advantageous in the case when there is only one input row, since all lookups can then be rendered and returned in streaming fashion. Release note (performance improvement): The execution engine can now short-circuit execution of lookup joins in more cases, which can decrease latency for queries with limits.
DrewKimball
force-pushed
the
streaming-lookup
branch
from
e8c846c to
d413fe7
Compare
There was a problem hiding this comment.
TFTR!
Reviewable status:
|
bors r+ |
|
Test failure looks like an unrelated flake. |
|
Build succeeded: |
Currently the
joinReaderOrderingStrategyimplementation buffers all looked uprows before matching them with input rows and emitting them. This is necessary
because the looked up rows may not be received in input order (which must be
maintained). However, rows that match the first input row can be emitted
immediately. In the case when there are many rows that match the first input
row, this can decrease overhead of the buffer. Additionally, this change can
allow a limit to be satisfied earlier, which can significantly decrease
latency. This is especially advantageous in the case when there is only one
input row, since all lookups can then be rendered and returned in streaming
fashion.
Release note (performance improvement): The execution engine can now
short-circuit execution of lookup joins in more cases, which can decrease
latency for queries with limits.