Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-22.1: opt: do not generate unnecessary cross-joins on join input #79504

Merged
merged 2 commits into from
Apr 6, 2022

Conversation

mgartner
Copy link
Collaborator

@mgartner mgartner commented Apr 6, 2022

Backport 2/2 commits from #79389.

/cc @cockroachdb/release


opt: do not generate unnecessary cross-joins on lookup join input

This commit fixes a bug that caused unnecessary cross-joins on the input
of lookup joins, causing both suboptimal query plans and incorrect query
results. The bug only affected lookup joins with lookup expressions.

Fixes #79384

Release note (bug fix): A bug has been fixed that caused the optimizer
to generate query plans with logically incorrect lookup joins. The bug
can only occur in queries with an inner join, e.g., t1 JOIN t2, if all
of the following are true:

  1. The join contains an equality condition between columns of both
    tables, e.g., t1.a = t2.a.
  2. A query filter or CHECK constraint constrains a column to a set
    of specific values, e.g., t2.b IN (1, 2, 3). In the case of a
    CHECK constraint, the column must be NOT NULL.
  3. A query filter or CHECK constraint constrains a column to a
    range, e.g., t2.c > 0. In the case of a CHECK constraint, the
    column must be NOT NULL.
  4. An index contains a column from each of the criteria above, e.g.,
    INDEX t2(a, b, c).
    This bug has been present since version 21.2.0.

opt: do not cross-join input of inverted semi-join

In #78685, we prevented GenerateLookupJoins from incorrect creating a
cross-join on the input of a semi-join, addressing #78681. This commit
addresses the same issue with GenerateInvertedJoins, which we
originally forgot to fix.

Informs #78681

Release note (bug fix): A bug has been fixed which caused the optimizer
to generate invalid query plans which could result in incorrect query
results. The bug, which has been present since version 21.1.0, can
appear if all of the following conditions are true:

  1. The query contains a semi-join, such as queries in the form
    SELECT * FROM a WHERE EXISTS (SELECT * FROM b WHERE a.a @> b.b).
  2. The inner table has a multi-column inverted index containing the
    inverted column in the filter.
  3. The index prefix columns are constrained to a set of values via the
    filter or a CHECK constraint, e.g., with an IN operator. In the
    case of a CHECK constraint, the column is NOT NULL.

Release justification: Fixes a correctness bug in the optimizer.

mgartner added 2 commits April 6, 2022 11:05
This commit fixes a bug that caused unnecessary cross-joins on the input
of lookup joins, causing both suboptimal query plans and incorrect query
results. The bug only affected lookup joins with lookup expressions.

Fixes cockroachdb#79384

Release note (bug fix): A bug has been fixed that caused the optimizer
to generate query plans with logically incorrect lookup joins. The bug
can only occur in queries with an inner join, e.g., `t1 JOIN t2`, if all
of the following are true:
  1. The join contains an equality condition between columns of both
     tables, e.g., `t1.a = t2.a`.
  2. A query filter or `CHECK` constraint constrains a column to a set
     of specific values, e.g., `t2.b IN (1, 2, 3)`. In the case of a
     `CHECK` constraint, the column must be `NOT NULL`.
  3. A query filter or `CHECK` constraint constrains a column to a
     range, e.g., `t2.c > 0`. In the case of a `CHECK` constraint, the
     column must be `NOT NULL`.
  4. An index contains a column from each of the criteria above, e.g.,
     `INDEX t2(a, b, c)`.
This bug has been present since version 21.2.0.
In cockroachdb#78685, we prevented `GenerateLookupJoins` from incorrect creating a
cross-join on the input of a semi-join, addressing cockroachdb#78681. This commit
addresses the same issue with `GenerateInvertedJoins`, which we
originally forgot to fix.

Informs cockroachdb#78681

Release note (bug fix): A bug has been fixed which caused the optimizer
to generate invalid query plans which could result in incorrect query
results. The bug, which has been present since version 21.1.0, can
appear if all of the following conditions are true:
  1. The query contains a semi-join, such as queries in the form
     `SELECT * FROM a WHERE EXISTS (SELECT * FROM b WHERE a.a @> b.b)`.
  2. The inner table has a multi-column inverted index containing the
     inverted column in the filter.
  3. The index prefix columns are constrained to a set of values via the
     filter or a `CHECK` constraint, e.g., with an `IN` operator. In the
     case of a `CHECK` constraint, the column is `NOT NULL`.
@mgartner mgartner requested review from rytaft and a team April 6, 2022 15:12
@mgartner mgartner requested a review from a team as a code owner April 6, 2022 15:12
@blathers-crl
Copy link

blathers-crl bot commented Apr 6, 2022

Thanks for opening a backport.

Please check the backport criteria before merging:

  • Patches should only be created for serious issues or test-only changes.
  • Patches should not break backwards-compatibility.
  • Patches should change as little code as possible.
  • Patches should not change on-disk formats or node communication protocols.
  • Patches should not add new functionality.
  • Patches must not add, edit, or otherwise modify cluster versions; or add version gates.
If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.
  • There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
  • The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
  • New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters.
  • The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.

Add a brief release justification to the body of your PR to justify this backport.

Some other things to consider:

  • What did we do to ensure that a user that doesn’t know & care about this backport, has no idea that it happened?
  • Will this work in a cluster of mixed patch versions? Did we test that?
  • If a user upgrades a patch version, uses this feature, and then downgrades, what happens?

@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@rytaft rytaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 8 of 8 files at r1, 3 of 3 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @mgartner)

@mgartner mgartner merged commit 9b51fc1 into cockroachdb:release-22.1 Apr 6, 2022
@mgartner mgartner deleted the backport22.1-79389 branch April 6, 2022 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants