Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: support array-flatten subqueries within UDFs #98879

Merged
merged 1 commit into from
Mar 21, 2023

Conversation

mgartner
Copy link
Collaborator

Array-flatten subqueries (e.g., ARRAY(SELECT a FROM t)) are now
supported within UDFs. They are now converted to a normal subquery with
a ScalarGroupBy if they exist within a UDF, even if they are
uncorrelated. This allows them to be executed without any changes to the
execbuilder or the evaluation logic of tree.Routine.

Fixes #98738

Release note: None

@mgartner mgartner requested a review from a team as a code owner March 17, 2023 19:08
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@rharding6373 rharding6373 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: Nice, thanks for the fix!

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @DrewKimball)

Copy link
Collaborator

@DrewKimball DrewKimball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: Nice!

Reviewed 9 of 9 files at r1, all commit messages.
Reviewable status: :shipit: complete! 2 of 0 LGTMs obtained (waiting on @mgartner)


pkg/sql/opt/norm/rules/scalar.opt line 389 at r1 (raw file):

# and correlated ones only over the types that array_agg supports.
#
# Note that optbuilder should have already verified that if the input is

Since you're at it, I think we can remove this bit since #35710 is fixed. Same for the check in optbuilder/scalar.go line 156.


pkg/sql/opt/exec/execbuilder/testdata/subquery line 566 at r1 (raw file):

      spans: FULL SCAN

# Case where the a correlated subquery contains an uncorrelated array-flatten

[nit] the a -> a

@mgartner mgartner force-pushed the 98738-array-flatten-udfs branch from a6d69ad to a8a3b75 Compare March 20, 2023 18:15
Copy link
Collaborator Author

@mgartner mgartner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 2 of 0 LGTMs obtained (waiting on @DrewKimball)


pkg/sql/opt/norm/rules/scalar.opt line 389 at r1 (raw file):

Previously, DrewKimball (Drew Kimball) wrote…

Since you're at it, I think we can remove this bit since #35710 is fixed. Same for the check in optbuilder/scalar.go line 156.

Done.


pkg/sql/opt/exec/execbuilder/testdata/subquery line 566 at r1 (raw file):

Previously, DrewKimball (Drew Kimball) wrote…

[nit] the a -> a

Done.

@mgartner
Copy link
Collaborator Author

bors r+

@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build failed:

Array-flatten subqueries (e.g., `ARRAY(SELECT a FROM t)`) are now
supported within UDFs. They are now converted to a normal subquery with
a ScalarGroupBy if they exist within a UDF, even if they are
uncorrelated. This allows them to be executed without any changes to the
execbuilder or the evaluation logic of `tree.Routine`.

Fixes cockroachdb#98738

Release note: None
@mgartner mgartner force-pushed the 98738-array-flatten-udfs branch from a8a3b75 to 99169e8 Compare March 21, 2023 13:33
Copy link
Collaborator Author

@mgartner mgartner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 2 stale) (waiting on @DrewKimball)


pkg/sql/opt/norm/rules/scalar.opt line 389 at r1 (raw file):

Previously, mgartner (Marcus Gartner) wrote…

Done.

This caused some test failures so I've reverted. I believe the check is still necessary.

Code quote:

# Note that optbuilder should have already verified that if the input is
# correlated, then we can array_agg over the input type.  Also note that the
# Max1Row operator we introduce is guaranteed to be eliminated as
# MakeArrayAggForFlatten will return a ScalarGroupBy.

@mgartner
Copy link
Collaborator Author

bors r+

@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build failed (retrying...):

@craig craig bot merged commit 77029b1 into cockroachdb:master Mar 21, 2023
@craig
Copy link
Contributor

craig bot commented Mar 21, 2023

Build succeeded:

@mgartner mgartner deleted the 98738-array-flatten-udfs branch March 21, 2023 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql: support ArrayFlatten subqueries within UDFs
4 participants