opt: internal error: no output column equivalent to 2 #85393
Assignees
Labels
branch-release-22.2
Used to mark GA and release blockers, technical advisories, and bugs for 22.2
C-bug
Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
O-community
Originated from the community
release-blocker
Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
T-sql-queries
SQL Queries Team
X-blathers-triaged
blathers was able to find an owner
Comments
bajinsheng
added
the
C-bug
|
Hello, I am Blathers. I am here to help you get the issue triaged. Hoot - a bug! Though bugs are the bane of my existence, rest assured the wretched thing will get the best of care here. I have CC'd a few people who may be able to assist you:
If we have not gotten back to your issue within a few business days, you can try the following:
🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan. |
blathers-crl
bot
added
O-community
mgartner
added
the
release-blocker
|
Hi @mgartner, please add branch-* labels to identify which branch(es) this release-blocker affects. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan. |
|
Thanks for the report! I've confirmed that this issue is not present in v22.1, v21.2, nor v21.1, so I've marked this as a release blocker. |
mgartner
changed the title
mgartner
added
the
branch-release-22.2
DrewKimball
added a commit
to DrewKimball/cockroach
that referenced
this issue
Aug 16, 2022
Project operators can only pass through their input ordering. However, the provided ordering may have to be remapped in order to ensure it only refers to output columns, since the `Project` can add and remove columns. The `Project` uses its `internalFDs` field to accomplish the remapping; these are constructed when the `Project` is added to the memo by combining the functional dependencies of the input and the projections. The problem occurs when transformation rules cause the input of the `Project` to "reveal" additional functional dependencies. For example, one side of a union may be eliminated and the FDs of the remaining side used in the result. This can cause the `Project` to output an ordering that is equivalent to the required ordering according to its own FDs, but which a parent operator cannot tell is equivalent because its FDs were calculated before the tranformation rule fired. This can cause panics later down the line when the provided ordering does not match up with the required ordering. In the following example, an exploration rule transforms the join into two joins unioned together, one over each disjunct. After the transformation, a normalization rule fires that removes the `t0.rowid IS NULL` side because rowids are non-null. This reveals the `t1.rowid = t0.rowid` FD, which later causes `t0.rowid` to be used in a provided ordering rather than `t1.rowid`. For the reasons mentioned above, this later causes a panic when a `Project` attempts to remap to the required `t1.rowid` ordering. ``` CREATE TABLE t0 (c0 INT); CREATE TABLE t1 (c0 INT); SELECT * FROM t0 CROSS JOIN t1 WHERE (t0.rowid IS NULL) OR (t1.rowid IN (t0.rowid)) ORDER BY t1.rowid; ``` This commit prevents the panic by making `Project` operators remap the input provided ordering to use columns from the required ordering (which are a subset of the output columns). This prevents the disparity between required and provided orderings that can cause panics down the line. In the example given above, the `t1.rowid` column would be chosen for the provided ordering because it is in the required ordering. Fixes cockroachdb#85393 Release note: None
craig bot
pushed a commit
that referenced
this issue
Aug 18, 2022
86055: sql: add index recommendation to execution_insights table r=j82w a=j82w Adds index recommendation to execution_insights table part of #81024 Release justification: Category 2: Bug fixes and low-risk updates to new functionality Release note (sql change): Adds index recommendation to execution_insights 86193: opt: use only required columns in provided ordering for project r=DrewKimball a=DrewKimball Project operators can only pass through their input ordering. However, the provided ordering may have to be remapped in order to ensure it only refers to output columns, since the `Project` can add and remove columns. The `Project` uses its `internalFDs` field to accomplish the remapping; these are constructed when the `Project` is added to the memo by combining the functional dependencies of the input and the projections. The problem occurs when transformation rules cause the input of the `Project` to "reveal" additional functional dependencies. For example, one side of a union may be eliminated and the FDs of the remaining side used in the result. This can cause the `Project` to output an ordering that is equivalent to the required ordering according to its own FDs, but which a parent operator cannot tell is equivalent because its FDs were calculated before the tranformation rule fired. This can cause panics later down the line when the provided ordering does not match up with the required ordering. In the following example, an exploration rule transforms the join into two joins unioned together, one over each disjunct. After the transformation, a normalization rule fires that removes the `t0.rowid IS NULL` side because rowids are non-null. This reveals the `t1.rowid = t0.rowid` FD, which later causes `t0.rowid` to be used in a provided ordering rather than `t1.rowid`. For the reasons mentioned above, this later causes a panic when a `Project` attempts to remap to the required `t1.rowid` ordering. ``` CREATE TABLE t0 (c0 INT); CREATE TABLE t1 (c0 INT); SELECT * FROM t0 CROSS JOIN t1 WHERE (t0.rowid IS NULL) OR (t1.rowid IN (t0.rowid)) ORDER BY t1.rowid; ``` This commit prevents the panic by making `Project` operators remap the input provided ordering to use columns from the required ordering (which are a subset of the output columns). This prevents the disparity between required and provided orderings that can cause panics down the line. In the example given above, the `t1.rowid` column would be chosen for the provided ordering because it is in the required ordering. Fixes #85393 Release note (bug fix): fixed a vulnerability in the optimizer that could cause a panic in rare cases when planning complex queries with `ORDER BY`. Release justification: fixes release blocker 86336: externalconn: grant `ALL` on `CREATE EXTERNAL CONNECTION` r=adityamaru a=adityamaru Previously, the user that created the external connection was not granted any privileges as the creator/owner of the object. This diff changes this by granting the user `ALL` privileges. When synthetic privileges have a concept of owners, we should also set the owner of the External Connection object to the creator. This can happen once #86181 is addressed. There was also a small bug in the grant/revoke code where external connection names that required to be wrapped in `""` egs: `"foo-kms"` were being incorrectly stringified when creating the synthetic privilege path. This resulted in: ``` CREATE EXTERNAL CONNECTION "foo-kms" AS ... GRANT USAGE ON EXTERNAL CONNECTION "foo-kms" TO baz ``` To fail with a "foo-kms" cannot be found, even though it was infact created before attempting the grant. Tests have been added to test this case. Fixes: #86261 Release note (bug fix): Users that create an external connection are now granted `ALL` privileges on the object. Release justification: bug fix for new functionality 86349: ttl: replace SPLIT AT with SplitTable in tests r=otan a=ecwall refs #85800 fixes #86376 SPLIT AT was causing occasional test failures if it did not asynchronously split the range before the TTL job started. SplitTable synchronously splits the range before the test starts the TTL job. Release justification: Fix test flake. Release note: None 86397: cluster-ui: added internal prefix to transaction workload insights view r=ericharmeling a=ericharmeling Previously, the connected component for transaction workload insights was using the selector for the sessions data slice to load the internal app name prefix. This isn't necessary; we can just use a constant, as is done on other pages. Fixes #86250. https://www.loom.com/share/18bf4edf46fb4b1dad2fe32f9d285dcd Release justification: bug fixes and low-risk updates to new functionality Release note: None Co-authored-by: j82w <[email protected]> Co-authored-by: DrewKimball <[email protected]> Co-authored-by: Aditya Maru <[email protected]> Co-authored-by: Evan Wall <[email protected]> Co-authored-by: Eric Harmeling <[email protected]>
DrewKimball
added a commit
to DrewKimball/cockroach
that referenced
this issue
Aug 24, 2022
Project operators can only pass through their input ordering. However, the provided ordering may have to be remapped in order to ensure it only refers to output columns, since the `Project` can add and remove columns. The `Project` uses its `internalFDs` field to accomplish the remapping; these are constructed when the `Project` is added to the memo by combining the functional dependencies of the input and the projections. The problem occurs when transformation rules cause the input of the `Project` to "reveal" additional functional dependencies. For example, one side of a union may be eliminated and the FDs of the remaining side used in the result. This can cause the `Project` to output an ordering that is equivalent to the required ordering according to its own FDs, but which a parent operator cannot tell is equivalent because its FDs were calculated before the tranformation rule fired. This can cause panics later down the line when the provided ordering does not match up with the required ordering. In the following example, an exploration rule transforms the join into two joins unioned together, one over each disjunct. After the transformation, a normalization rule fires that removes the `t0.rowid IS NULL` side because rowids are non-null. This reveals the `t1.rowid = t0.rowid` FD, which later causes `t0.rowid` to be used in a provided ordering rather than `t1.rowid`. For the reasons mentioned above, this later causes a panic when a `Project` attempts to remap to the required `t1.rowid` ordering. ``` CREATE TABLE t0 (c0 INT); CREATE TABLE t1 (c0 INT); SELECT * FROM t0 CROSS JOIN t1 WHERE (t0.rowid IS NULL) OR (t1.rowid IN (t0.rowid)) ORDER BY t1.rowid; ``` This commit prevents the panic by making `Project` operators remap the input provided ordering to use columns from the required ordering (which are a subset of the output columns). This prevents the disparity between required and provided orderings that can cause panics down the line. In the example given above, the `t1.rowid` column would be chosen for the provided ordering because it is in the required ordering. Fixes cockroachdb#85393 Release note (bug fix): fixed a vulnerability in the optimizer that could cause a panic in rare cases when planning complex queries with `ORDER BY`. Release justification: low-risk bug fix
DrewKimball
added a commit
to DrewKimball/cockroach
that referenced
this issue
Aug 24, 2022
Project operators can only pass through their input ordering. However, the provided ordering may have to be remapped in order to ensure it only refers to output columns, since the `Project` can add and remove columns. The `Project` uses its `internalFDs` field to accomplish the remapping; these are constructed when the `Project` is added to the memo by combining the functional dependencies of the input and the projections. The problem occurs when transformation rules cause the input of the `Project` to "reveal" additional functional dependencies. For example, one side of a union may be eliminated and the FDs of the remaining side used in the result. This can cause the `Project` to output an ordering that is equivalent to the required ordering according to its own FDs, but which a parent operator cannot tell is equivalent because its FDs were calculated before the tranformation rule fired. This can cause panics later down the line when the provided ordering does not match up with the required ordering. In the following example, an exploration rule transforms the join into two joins unioned together, one over each disjunct. After the transformation, a normalization rule fires that removes the `t0.rowid IS NULL` side because rowids are non-null. This reveals the `t1.rowid = t0.rowid` FD, which later causes `t0.rowid` to be used in a provided ordering rather than `t1.rowid`. For the reasons mentioned above, this later causes a panic when a `Project` attempts to remap to the required `t1.rowid` ordering. ``` CREATE TABLE t0 (c0 INT); CREATE TABLE t1 (c0 INT); SELECT * FROM t0 CROSS JOIN t1 WHERE (t0.rowid IS NULL) OR (t1.rowid IN (t0.rowid)) ORDER BY t1.rowid; ``` This commit prevents the panic by making `Project` operators remap the input provided ordering to use columns from the required ordering (which are a subset of the output columns). This prevents the disparity between required and provided orderings that can cause panics down the line. In the example given above, the `t1.rowid` column would be chosen for the provided ordering because it is in the required ordering. Fixes cockroachdb#85393 Release note (bug fix): fixed a vulnerability in the optimizer that could cause a panic in rare cases when planning complex queries with `ORDER BY`. Release justification: low-risk bug fix
DrewKimball
added a commit
to DrewKimball/cockroach
that referenced
this issue
Apr 6, 2023
It is possible for some functional-dependency information to be visible to a child operator but invisible to its parent. This could previously cause panics when a child provided an ordering that could be proven to satisfy the required ordering with the child FDs, but not with the parent's FDs. This patch adds a step to the logic that builds provided orderings that ensures a provided ordering can be proven to respect the required ordering without needing additional FD information. This ensures that a parent never needs to know its child's FDs in order to prove that the provided ordering is correct. The extra step is a no-op in the common case when the provided ordering can already be proven to respect the required ordering. Informs cockroachdb#85393 Informs cockroachdb#87806 Fixes cockroachdb#96288 Release note (bug fix): Fixed a rare internal error in the optimizer that has existed since before version 22.1, which could occur while enforcing orderings between SQL operators.
DrewKimball
added a commit
to DrewKimball/cockroach
that referenced
this issue
Apr 10, 2023
It is possible for some functional-dependency information to be visible to a child operator but invisible to its parent. This could previously cause panics when a child provided an ordering that could be proven to satisfy the required ordering with the child FDs, but not with the parent's FDs. This patch adds a step to the logic that builds provided orderings that ensures a provided ordering can be proven to respect the required ordering without needing additional FD information. This ensures that a parent never needs to know its child's FDs in order to prove that the provided ordering is correct. The extra step is a no-op in the common case when the provided ordering can already be proven to respect the required ordering. Informs cockroachdb#85393 Informs cockroachdb#87806 Fixes cockroachdb#96288 Release note (bug fix): Fixed a rare internal error in the optimizer that has existed since before version 22.1, which could occur while enforcing orderings between SQL operators.
craig bot
pushed a commit
that referenced
this issue
Apr 10, 2023
100533: workload: jitter the teardown of connections to prevent thundering herd r=sean- a=sean- This change upgrades workload's use of pgx from v4 to v5 in order to allow jittering the teardown of connections. This change sets a max connection age of 5min and jitters the teardown by 30s. Upgrading to pgx v5 also adds non-blocking pgxpool connection acquisition. workload: add flags to manage the age and lifecycle of connection pool Add flags to all workload types to specify: * the max connection age: `--max-conn-lifetime duration` * the max connection age jitter: `--max-conn-lifetime-jitter duration` * the max connection idle time: `--max-conn-idle-time duration` * the connection health check interval: `--conn-healthcheck-period duration` * the min number of connections in the pool: `--min-conns int` workload: add support for remaining pgx query modes Add support for pgx.QueryExecModeCacheDescribe and pgx.QueryExecModeDescribeExec. Previously, only three of the five query modes were available. workload: fix race condition when recording histogram data Release note (cli change): workload jitters teardown of connections to prevent thundering herd impacting P99 latency results. Release note (cli change): workload utility now has flags to tune the connection pool used for testing. See `--conn-healthcheck-period`, `--min-conns`, and the `--max-conn-*` flags for details. Release note (cli change): workload now supports every [PostgreSQL query mode](https://github.com/jackc/pgx/blob/fa5fbed497bc75acee05c1667a8760ce0d634cba/conn.go#L167-L182) available via the underlying pgx driver. 100776: opt: fix ordering-related optimizer panics r=DrewKimball a=DrewKimball It is possible for some functional-dependency information to be visible to a child operator but invisible to its parent. This could previously cause panics when a child provided an ordering that could be proven to satisfy the required ordering with the child FDs, but not with the parent's FDs. This patch adds a step to the logic that builds provided orderings that ensures a provided ordering can be proven to respect the required ordering without needing additional FD information. This ensures that a parent never needs to know its child's FDs in order to prove that the provided ordering is correct. The extra step is a no-op in the common case when the provided ordering can already be proven to respect the required ordering. Informs #85393 Informs #87806 Fixes #96288 Release note (bug fix): Fixed a rare internal error in the optimizer that has existed since before version 22.1, which could occur while enforcing orderings between SQL operators. 101076: autoconfig: prevent a data race in TestAutoConfig r=adityamaru a=knz Needed for #101069. The calls to Peek and Pop can run concurrently. Release note: None Epic: CRDB-23559 101078: roachtest: move copyfrom test suite to SQL Queries r=srosenberg a=nvanbenschoten See https://cockroachlabs.slack.com/archives/C0168LW5THS/p1679508254391039. Epic: None Release note: None Co-authored-by: Sean Chittenden <[email protected]> Co-authored-by: Drew Kimball <[email protected]> Co-authored-by: Raphael 'kena' Poss <[email protected]> Co-authored-by: Nathan VanBenschoten <[email protected]>
blathers-crl bot
pushed a commit
that referenced
this issue
Apr 10, 2023
It is possible for some functional-dependency information to be visible to a child operator but invisible to its parent. This could previously cause panics when a child provided an ordering that could be proven to satisfy the required ordering with the child FDs, but not with the parent's FDs. This patch adds a step to the logic that builds provided orderings that ensures a provided ordering can be proven to respect the required ordering without needing additional FD information. This ensures that a parent never needs to know its child's FDs in order to prove that the provided ordering is correct. The extra step is a no-op in the common case when the provided ordering can already be proven to respect the required ordering. Informs #85393 Informs #87806 Fixes #96288 Release note (bug fix): Fixed a rare internal error in the optimizer that has existed since before version 22.1, which could occur while enforcing orderings between SQL operators.
This was referenced Apr 10, 2023
blathers-crl bot
pushed a commit
that referenced
this issue
Apr 12, 2023
It is possible for some functional-dependency information to be visible to a child operator but invisible to its parent. This could previously cause panics when a child provided an ordering that could be proven to satisfy the required ordering with the child FDs, but not with the parent's FDs. This patch adds a step to the logic that builds provided orderings that ensures a provided ordering can be proven to respect the required ordering without needing additional FD information. This ensures that a parent never needs to know its child's FDs in order to prove that the provided ordering is correct. The extra step is a no-op in the common case when the provided ordering can already be proven to respect the required ordering. Informs #85393 Informs #87806 Fixes #96288 Release note (bug fix): Fixed a rare internal error in the optimizer that has existed since before version 22.1, which could occur while enforcing orderings between SQL operators.
DrewKimball
added a commit
to DrewKimball/cockroach
that referenced
this issue
Oct 25, 2023
It is possible for some functional-dependency information to be visible to a child operator but invisible to its parent. This could previously cause panics when a child provided an ordering that could be proven to satisfy the required ordering with the child FDs, but not with the parent's FDs. This patch adds a step to the logic that builds provided orderings that ensures a provided ordering can be proven to respect the required ordering without needing additional FD information. This ensures that a parent never needs to know its child's FDs in order to prove that the provided ordering is correct. The extra step is a no-op in the common case when the provided ordering can already be proven to respect the required ordering. Informs cockroachdb#85393 Informs cockroachdb#87806 Fixes cockroachdb#96288 Release note (bug fix): Fixed a rare internal error in the optimizer that has existed since before version 22.1, which could occur while enforcing orderings between SQL operators.
DrewKimball
added a commit
to DrewKimball/cockroach
that referenced
this issue
Oct 26, 2023
It is possible for some functional-dependency information to be visible to a child operator but invisible to its parent. This could previously cause panics when a child provided an ordering that could be proven to satisfy the required ordering with the child FDs, but not with the parent's FDs. This patch adds a step to the logic that builds provided orderings that ensures a provided ordering can be proven to respect the required ordering without needing additional FD information. This ensures that a parent never needs to know its child's FDs in order to prove that the provided ordering is correct. The extra step is a no-op in the common case when the provided ordering can already be proven to respect the required ordering. Informs cockroachdb#85393 Informs cockroachdb#87806 Fixes cockroachdb#96288 Release note (bug fix): Fixed a rare internal error in the optimizer that has existed since before version 22.1, which could occur while enforcing orderings between SQL operators.
blathers-crl bot
pushed a commit
that referenced
this issue
Nov 1, 2023
It is possible for some functional-dependency information to be visible to a child operator but invisible to its parent. This could previously cause panics when a child provided an ordering that could be proven to satisfy the required ordering with the child FDs, but not with the parent's FDs. This patch adds a step to the logic that builds provided orderings that ensures a provided ordering can be proven to respect the required ordering without needing additional FD information. This ensures that a parent never needs to know its child's FDs in order to prove that the provided ordering is correct. The extra step is a no-op in the common case when the provided ordering can already be proven to respect the required ordering. Informs #85393 Informs #87806 Fixes #96288 Release note (bug fix): Fixed a rare internal error in the optimizer that has existed since before version 22.1, which could occur while enforcing orderings between SQL operators.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the problem
An unexpected internal error.
To Reproduce
Expected behavior
No error.
Additional data / screenshots
Environment:
cockroach sql]Additional context
Jira issue: CRDB-18224
The text was updated successfully, but these errors were encountered: