Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
78224: sql: use soft and hard limits in lookup and index joins r=yuzefovich a=yuzefovich Previously, during the execution of lookup and index joins we completely ignored the soft and hard limits and, instead, always fetched up to the memory-based limit (which depends on the type of the join). This could lead to having to read many more rows from the input and then looking up many more rows from the KV layer than necessary. This is now fixed by plumbing the soft and hard limits from the optimizer and using them when sizing the batches of input rows to perform lookup for. For index joins we know for sure that every input row will get a looked up row; however, for lookup joins an input row might result in a miss. To work around this we use a simple heuristic for determining the limit-hint-based size of input batches: for the first batch we use the limit hint as is, for the second batch we use 10x of the original hint, and for the third and all consequent batches we disable the limiting behavior altogether. Fixes: cockroachdb#77715. Release note (bug fix): CockroachDB might now fetch less rows when performing lookup and index joins on the queries with LIMIT clause. 78376: colexecjoin: optimize building output on the left in cross joiner r=yuzefovich a=yuzefovich This commit updates the way we're building output in the cross joiner from the left input (also used by the merge joiner when building from the buffered group). There, we need to repeat a single tuple `toAppend` times, so we do it in a loop. This commit adds the optimization of using `Bytes.Copy` for the bytes-like types as well as BCE for sliceable types. ``` name old speed new speed delta CrossJoiner/spillForced=false/type=INNER/rows=1-24 1.01MB/s ± 1% 1.00MB/s ± 2% -1.18% (p=0.013 n=10+10) CrossJoiner/spillForced=false/type=INNER/rows=16-24 207MB/s ± 0% 205MB/s ± 2% -0.76% (p=0.023 n=10+10) CrossJoiner/spillForced=false/type=INNER/rows=256-24 6.77GB/s ± 1% 7.78GB/s ± 0% +14.92% (p=0.000 n=10+8) CrossJoiner/spillForced=false/type=INNER/rows=2048-24 8.87GB/s ± 1% 10.33GB/s ± 0% +16.55% (p=0.000 n=9+9) CrossJoiner/spillForced=false/type=INNER/rows=8192-24 8.92GB/s ± 1% 10.39GB/s ± 1% +16.52% (p=0.000 n=10+10) ``` ``` name old speed new speed delta MergeJoiner/rows=32-24 34.7MB/s ± 2% 34.6MB/s ± 3% ~ (p=0.896 n=10+10) MergeJoiner/rows=512-24 94.7MB/s ± 3% 94.6MB/s ± 2% ~ (p=0.619 n=10+9) MergeJoiner/rows=4096-24 235MB/s ± 1% 233MB/s ± 1% -0.94% (p=0.004 n=9+10) MergeJoiner/rows=32768-24 341MB/s ± 3% 340MB/s ± 2% ~ (p=0.315 n=10+10) MergeJoiner/oneSideRepeat-rows=32-24 44.1MB/s ± 2% 44.1MB/s ± 3% ~ (p=0.839 n=10+10) MergeJoiner/oneSideRepeat-rows=512-24 252MB/s ± 2% 262MB/s ± 2% +3.93% (p=0.000 n=10+10) MergeJoiner/oneSideRepeat-rows=4096-24 904MB/s ± 1% 953MB/s ± 2% +5.50% (p=0.000 n=10+10) MergeJoiner/oneSideRepeat-rows=32768-24 1.45GB/s ± 1% 1.53GB/s ± 2% +5.43% (p=0.000 n=10+9) MergeJoiner/bothSidesRepeat-rows=32-24 27.1MB/s ± 1% 27.2MB/s ± 2% ~ (p=0.722 n=10+10) MergeJoiner/bothSidesRepeat-rows=512-24 124MB/s ± 3% 127MB/s ± 2% +2.55% (p=0.001 n=10+10) MergeJoiner/bothSidesRepeat-rows=4096-24 150MB/s ± 1% 152MB/s ± 1% +1.25% (p=0.000 n=10+10) MergeJoiner/bothSidesRepeat-rows=32768-24 84.8MB/s ± 1% 86.4MB/s ± 0% +1.93% (p=0.000 n=10+10) ``` Release note: None Co-authored-by: Yahor Yuzefovich <[email protected]>
- Loading branch information