Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opt: internal error due to outer variable in left side of join #46151

Closed
rytaft opened this issue Mar 16, 2020 · 1 comment · Fixed by #46153
Closed

opt: internal error due to outer variable in left side of join #46151

rytaft opened this issue Mar 16, 2020 · 1 comment · Fixed by #46153
Assignees
Labels
A-sql-optimizer SQL logical planning and optimizations.
Milestone

Comments

@rytaft
Copy link
Collaborator

rytaft commented Mar 16, 2020

To reproduce, create a database with the TPC-C schema, then run the following query:

SELECT
	subq_0.c0 AS c0,
	subq_0.c0 AS c1,
	subq_0.c0 AS c2,
	subq_0.c0 AS c3,
	CASE
	WHEN subq_0.c1 IS NOT NULL THEN subq_0.c0
	ELSE subq_0.c0
	END
		AS c4
FROM
	(
		SELECT
			ref_0.h_data AS c0,
			(SELECT i_name FROM item LIMIT 1 OFFSET 6) AS c1
		FROM
			history AS ref_0
			INNER JOIN order_line AS ref_1 ON
					(ref_0.h_data = ref_1.ol_dist_info)
		WHERE
			(
				EXISTS(
					SELECT
						ref_2.h_amount AS c0,
						ref_2.h_c_w_id AS c1,
						ref_2.h_c_d_id AS c2
					FROM
						history AS ref_2
					WHERE
						(ref_2.h_data IS NOT NULL)
						AND (ref_1.ol_dist_info IS NOT NULL)
				)
			)
			OR (
					(SELECT ol_i_id FROM order_line LIMIT 1 OFFSET 6)
					IS NOT NULL
				)
		LIMIT
			2
	)
		AS subq_0
WHERE
	subq_0.c1 IS NULL
LIMIT
	6;

It crashes with this error:

Error: pq: internal error: cannot map variable 19 to an indexed var
DETAIL: stack trace:
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:149: indexedVar()
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:134: buildVariable()
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:91: buildScalar()
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:225: buildComparison()
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:91: buildScalar()
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:212: buildBoolean()
github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder/scalar.go:91: buildScalar()
...

Thanks to the Apollo team at Georgia Tech for finding this issue.

@rytaft rytaft added the A-sql-optimizer SQL logical planning and optimizations. label Mar 16, 2020
@rytaft rytaft added this to the 20.1 milestone Mar 16, 2020
@rytaft rytaft self-assigned this Mar 16, 2020
rytaft added a commit to rytaft/cockroach that referenced this issue Mar 16, 2020
Prior to this commit, it was possible for a correlated subquery to
go undetected if it was buried in a complex filter. In particular,
a filter of the form:

  <correlated subquery> OR <non-correlated subquery>

would incorrectly be marked as *not* containing a correlated subquery.
This was because although the logical property HasCorrelatedSubquery
was initially set to true upon encountering the first (correlated)
subquery, the left-to-right recursive traversal of the OR expression
caused HasCorrelatedSubquery to be overwritten to false upon encountering
the second (non-correlated) subquery.

This commit fixes the issue by never overwriting HasCorrelatedSubquery
to false.

Fixes cockroachdb#46151

Release note (bug fix): Fixed an internal error that could occur in the
optimizer when a WHERE filter contained at least one correlated subquery
and one non-correlated subquery.

Release justification: This bug fix falls into the category "low risk,
high benefit changes to existing functionality".
@rytaft
Copy link
Collaborator Author

rytaft commented Mar 16, 2020

Also thanks to @RaduBerinde for narrowing down the root cause and the updates to check_expr.go.

@rytaft rytaft changed the title opt: crash due to outer variable in left side of join opt: internal error due to outer variable in left side of join Mar 16, 2020
craig bot pushed a commit that referenced this issue Mar 16, 2020
46153: opt: fix detection of correlated subqueries in complex filters r=rytaft a=rytaft

Prior to this commit, it was possible for a correlated subquery to
go undetected if it was buried in a complex filter. In particular,
a filter of the form:
```
  <correlated subquery> OR <non-correlated subquery>
```
would incorrectly be marked as *not* containing a correlated subquery.
This was because although the logical property `HasCorrelatedSubquery`
was initially set to true upon encountering the first (correlated)
subquery, the left-to-right recursive traversal of the `OR` expression
caused `HasCorrelatedSubquery` to be overwritten to false upon encountering
the second (non-correlated) subquery.

This commit fixes the issue by never overwriting `HasCorrelatedSubquery`
to false.

Fixes #46151

Release note (bug fix): Fixed an internal error that could occur in the
optimizer when a WHERE filter contained at least one correlated subquery
and one non-correlated subquery.

Release justification: This bug fix falls into the category "low risk,
high benefit changes to existing functionality".

46160: vendor: Bump pebble to c50a7b1164d9b1a971f1729d84ff9cdb7e987496 r=itsbilal a=itsbilal

Picks up these changes:
 - iterator: Copy iterValue before Prev() on internal iterator
 - db: add a section on RocksDB Compatibility
 - db: rework doc comment on Iterator.SeekPrefixGE

Release justification: Bug fixes.
Release note: None

Co-authored-by: Rebecca Taft <[email protected]>
Co-authored-by: Bilal Akhtar <[email protected]>
@craig craig bot closed this as completed in c1c285e Mar 16, 2020
rytaft added a commit to rytaft/cockroach that referenced this issue Mar 16, 2020
Prior to this commit, it was possible for a correlated subquery to
go undetected if it was buried in a complex filter. In particular,
a filter of the form:

  <correlated subquery> OR <non-correlated subquery>

would incorrectly be marked as *not* containing a correlated subquery.
This was because although the logical property HasCorrelatedSubquery
was initially set to true upon encountering the first (correlated)
subquery, the left-to-right recursive traversal of the OR expression
caused HasCorrelatedSubquery to be overwritten to false upon encountering
the second (non-correlated) subquery.

This commit fixes the issue by never overwriting HasCorrelatedSubquery
to false.

Fixes cockroachdb#46151

Release note (bug fix): Fixed an internal error that could occur in the
optimizer when a WHERE filter contained at least one correlated subquery
and one non-correlated subquery.

Release justification: This bug fix falls into the category "low risk,
high benefit changes to existing functionality".
rytaft added a commit to rytaft/cockroach that referenced this issue Mar 16, 2020
Prior to this commit, it was possible for a correlated subquery to
go undetected if it was buried in a complex filter. In particular,
a filter of the form:

  <correlated subquery> OR <non-correlated subquery>

would incorrectly be marked as *not* containing a correlated subquery.
This was because although the logical property HasCorrelatedSubquery
was initially set to true upon encountering the first (correlated)
subquery, the left-to-right recursive traversal of the OR expression
caused HasCorrelatedSubquery to be overwritten to false upon encountering
the second (non-correlated) subquery.

This commit fixes the issue by never overwriting HasCorrelatedSubquery
to false.

Fixes cockroachdb#46151

Release note (bug fix): Fixed an internal error that could occur in the
optimizer when a WHERE filter contained at least one correlated subquery
and one non-correlated subquery.

Release justification: This bug fix falls into the category "low risk,
high benefit changes to existing functionality".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-sql-optimizer SQL logical planning and optimizations.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant