Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: support ArrayFlatten subqueries within UDFs #98738

Closed
rharding6373 opened this issue Mar 16, 2023 · 1 comment · Fixed by #98879
Closed

sql: support ArrayFlatten subqueries within UDFs #98738

rharding6373 opened this issue Mar 16, 2023 · 1 comment · Fixed by #98879
Assignees
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. GA-blocker T-sql-queries SQL Queries Team

Comments

@rharding6373
Copy link
Collaborator

rharding6373 commented Mar 16, 2023

The following UDF is able to be created, but unexpectedly fails because the subquery was unable to be planned lazily:

CREATE FUNCTION f() RETURNS INT LANGUAGE SQL AS $$ SELECT array(SELECT substr('ABCDE',2,1)); SELECT 1;$$;
SELECT f();
ERROR: internal error: expected subquery to be lazily planned as a routine
SQLSTATE: XX000
DETAIL: stack trace:
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:1039: expectedLazyRoutineError()
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:1019: func2()
github.com/cockroachdb/cockroach/pkg/sql/routine.go:137: Start()
github.com/cockroachdb/cockroach/pkg/sql/routine.go:39: EvalRoutineExpr()
github.com/cockroachdb/cockroach/pkg/sql/sem/eval/expr.go:625: EvalRoutineExpr()
github.com/cockroachdb/cockroach/bazel-out/darwin-fastbuild/bin/pkg/sql/sem/tree/eval_expr_generated.go:346: Eval()
github.com/cockroachdb/cockroach/pkg/sql/sem/eval/expr.go:26: Expr()
github.com/cockroachdb/cockroach/pkg/sql/values.go:80: startExec()
github.com/cockroachdb/cockroach/pkg/sql/plan.go:524: func2()
github.com/cockroachdb/cockroach/pkg/sql/walk.go:112: func1()
github.com/cockroachdb/cockroach/pkg/sql/walk.go:299: visitInternal()
github.com/cockroachdb/cockroach/pkg/sql/walk.go:79: visit()
github.com/cockroachdb/cockroach/pkg/sql/walk.go:43: walkPlan()
github.com/cockroachdb/cockroach/pkg/sql/plan.go:527: startExec()
github.com/cockroachdb/cockroach/pkg/sql/plan_node_to_row_source.go:173: Start()
github.com/cockroachdb/cockroach/pkg/sql/colexec/columnarizer.go:184: Init()
github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:94: init()
github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92: CatchVectorizedRuntimeError()
github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:103: Init()
github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:248: func1()
github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92: CatchVectorizedRuntimeError()
github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:247: init()
github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:287: Run()
github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:309: Run()
github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:864: Run()
github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1835: PlanAndRun()
github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1577: func2()
github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1580: PlanAndRunAll()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1795: execWithDistSQLEngine()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1378: dispatchToExecutionEngine()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:785: execStmtInOpenState()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:137: func1()

HINT: You have encountered an unexpected error.

Jira issue: CRDB-25554

@rharding6373 rharding6373 added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Mar 16, 2023
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Mar 16, 2023
@mgartner mgartner self-assigned this Mar 16, 2023
@blathers-crl
Copy link

blathers-crl bot commented Mar 16, 2023

Hi @mgartner, please add branch-* labels to identify which branch(es) this release-blocker affects.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@mgartner mgartner added the branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 label Mar 17, 2023
@mgartner mgartner changed the title sql: udf expectedLazyRoutineError sql: support ArrayFlatten subqueries within UDFs Mar 17, 2023
mgartner added a commit to mgartner/cockroach that referenced this issue Mar 17, 2023
Array-flatten subqueries (e.g., `ARRAY(SELECT a FROM t)`) are now
supported within UDFs. They are now converted to a normal subquery with
a ScalarGroupBy if they exist within a UDF, even if they are
uncorrelated. This allows them to be executed without any changes to the
execbuilder or the evaluation logic of `tree.Routine`.

Fixes cockroachdb#98738

Release note: None
mgartner added a commit to mgartner/cockroach that referenced this issue Mar 20, 2023
Array-flatten subqueries (e.g., `ARRAY(SELECT a FROM t)`) are now
supported within UDFs. They are now converted to a normal subquery with
a ScalarGroupBy if they exist within a UDF, even if they are
uncorrelated. This allows them to be executed without any changes to the
execbuilder or the evaluation logic of `tree.Routine`.

Fixes cockroachdb#98738

Release note: None
craig bot pushed a commit that referenced this issue Mar 21, 2023
98527: sql: deprecate gossip-based physical planning r=dt a=dt

Previously system tenants and secondary tenants used differnt physical planning implementations, with the system tenant and only the system tenant using nodeIDs while other tenants used the instance table. This unifies those implementations such that all tenants use NodeIDs if running in mixed mode and use the instance table if not.

Release note: none.
Epic: CRDB-16910

98879: sql: support array-flatten subqueries within UDFs r=mgartner a=mgartner

Array-flatten subqueries (e.g., `ARRAY(SELECT a FROM t)`) are now
supported within UDFs. They are now converted to a normal subquery with
a ScalarGroupBy if they exist within a UDF, even if they are
uncorrelated. This allows them to be executed without any changes to the
execbuilder or the evaluation logic of `tree.Routine`.

Fixes #98738

Release note: None


98988: ui: add badges for filter elements r=maryliag a=maryliag

Adds badges for each of the selected filters on SQL Activity and
Insights pages.

Part Of #98891

https://www.loom.com/share/e7417209fc704d71b2733f58609fb4de

<img width="1160" alt="Screenshot 2023-03-19 at 1 30 49 PM" src="https://user-images.githubusercontent.com/1017486/226195412-03d3ef58-5d6d-4c24-84f1-6a55b952f5c0.png">

Release note (ui change): Adds badges for each selected
filter on SQL Activity and Insights pages.

99042: sql: added indexes to system.statement_statistics, system.transaction_statistics r=ericharmeling a=ericharmeling

Fixes #98624.

This commit adds indexes on new computed columns to the system.statement_statistics and system.transaction_statistics tables.

Epic: none

Release note: None

99054: pkg/ccl: Unskip TestTenantStatusAPI/tenant_ranges/pagination r=dhartunian a=abarganier

Fixes: #92979

Previously, in #97386, we skipped test_tenant_ranges_pagination because it was marked as flaky.

The test makes a request for a single range and expects an offset of `1` back. It then uses this offset to request a second range, and expects an offset of `2`. This means that the test requires at least 3 ranges to exist on the tenant.

The test was flaking on the assertion that the offset returned by the second request came back as `2`. Instead, it was flaking when the offset came back as `0`, which signifies that there are no more ranges to process.

We learned that the tenant create process has an asycnhronous splitting of ranges that occurs, which is what would lead to this sporadic scenario where not enough ranges existed (yet) for the test to succeed.

This patch updates the test with a `testutils.SucceedsSoon` clause that checks first that `crdb_internal.ranges` contains at least 3 ranges, prior to making the second request. This should provide sufficient time for the range split queue to be processed and eliminate the vast majority of these test flakes.

Release note: none

99119: kvserver: mark in-flight storage checkpoints r=tbg a=pavelkalinnikov

This commit makes it so that the consistency checker checkpoints are first created in a "_pending" folder, and only after completion are atomically renamed to the intended directory name. This is to help distinguish valid checkpoints from the ones that weren't finalized (for example, when the node crashed in the meantime).

Part of #81819
Epic: none
Release note (ops change): checkpoint directories (that can be created in the rare event of range inconsistency) are now clearly indicated as pending until they are fully populated. This is to help operators to distinguish valid checkpoints from corrupted ones.

Co-authored-by: David Taylor <[email protected]>
Co-authored-by: Marcus Gartner <[email protected]>
Co-authored-by: maryliag <[email protected]>
Co-authored-by: Eric Harmeling <[email protected]>
Co-authored-by: Alex Barganier <[email protected]>
Co-authored-by: Pavel Kalinnikov <[email protected]>
@craig craig bot closed this as completed in 99169e8 Mar 21, 2023
@mgartner mgartner moved this to Done in SQL Queries Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. GA-blocker T-sql-queries SQL Queries Team
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants