Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql,kv: power ColIndexJoin by COL_BATCH_RESPONSE scan format #94807

Open
yuzefovich opened this issue Jan 6, 2023 · 0 comments
Open

sql,kv: power ColIndexJoin by COL_BATCH_RESPONSE scan format #94807

yuzefovich opened this issue Jan 6, 2023 · 0 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team

Comments

@yuzefovich
Copy link
Member

yuzefovich commented Jan 6, 2023

The initial work on the KV projection pushdown will only utilize the newly-introduced COL_BATCH_RESPONSE scan format for simple scans (i.e. TableReaders). We should extend that also to index joins since we already have the vectorized ColIndexJoin (unlike the lookup joins that still go through the row-by-row engine).

The main contribution here would be to teach the Streamer to use the new scan format.

Also an important question that will need to be answered as part of this item is whether we squash coldata.Batches from multiple requests into one or not. This is important since index joins read only a single row, so setting up the whole pushdown pipeline for each request separately can lead to significant overhead. One idea that was mentioned previously was to introduce a new request type (something like BatchedScanRequest) which would contain key spans that need to be scanned and can be returned as a single response. For index joins we don't care to know which particular input row resulted in which response row.

Jira issue: CRDB-23150

@yuzefovich yuzefovich added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Jan 6, 2023
@yuzefovich yuzefovich self-assigned this Jan 6, 2023
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Jan 6, 2023
@yuzefovich yuzefovich removed their assignment Mar 14, 2023
@mgartner mgartner moved this to New Backlog in SQL Queries Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team
Projects
Status: Backlog
Development

No branches or pull requests

1 participant