release-22.2: opt: hoist uncorrelated equality subqueries

[mgartner]

Backport 1/1 commits from #100881.

/cc @cockroachdb/release

---

opt: hoist uncorrelated equality subqueries

Subqueries that are in equality expressions with a variable are now
hoisted. When these expressions exist in a filter, hoisting the subquery
can allow the main query to plan a lookup join, rather than an
inefficient full-table scan.

For example, consider the table and query:

CREATE TABLE t (
  a INT,
  INDEX (a)
);

SELECT * FROM t WHERE a = (SELECT max(a) FROM t);

Prior to this commit, the query plan for this query required a full
table scan:

select
 ├── columns: a:1
 ├── scan t@t_a_idx
 │    ├── columns: a:1
 │    └── constraint: /1/2: (/NULL - ]
 └── filters
      └── eq
           ├── a:1
           └── subquery
                └── scalar-group-by
                     ├── columns: max:9
                     ├── scan t@t_a_idx,rev
                     │    ├── columns: a:5
                     │    ├── constraint: /5/6: (/NULL - ]
                     │    └── limit: 1(rev)
                     └── aggregations
                          └── const-agg [as=max:9, outer=(5)]
                               └── a:5

By hoisting the subquery, the full table scan is replaced with a lookup
join:

project
 ├── columns: a:1
 └── inner-join (lookup t@t_a_idx)
      ├── columns: a:1 max:9
      ├── key columns: [9] = [1]
      ├── scalar-group-by
      │    ├── columns: max:9
      │    ├── scan t@t_a_idx,rev
      │    │    ├── columns: a:5
      │    │    ├── constraint: /5/6: (/NULL - ]
      │    │    └── limit: 1(rev)
      │    └── aggregations
      │         └── const-agg [as=max:9, outer=(5)]
      │              └── a:5
      └── filters (true)

This hoisting is disabled by default, but can be enabled by setting the
optimizer_hoist_uncorrelated_equality_subqueries session setting to
true.

Fixes #83392
Informs #51820
Informs #93829
Informs #100855

Release note (performance improvement): Queries that have subqueries in
equality expressions are now more efficiently planned by the optimizer when
optimizer_hoist_uncorrelated_equality_subqueries is set to true.

---

Release justification: This is an optimizer improvement that is disabled by
default to avoid negatively affecting query plans.

[cockroach-teamcity]

This change is 

[blathers-crl]

Thanks for opening a backport.

Please check the backport criteria before merging:

• Patches should only be created for serious issues or test-only changes.
• Patches should not break backwards-compatibility.
• Patches should change as little code as possible.
• Patches should not change on-disk formats or node communication protocols.
• Patches should not add new functionality.
• Patches must not add, edit, or otherwise modify cluster versions; or add version gates.

If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.

• There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
• The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
• New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters.
• The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.

Add a brief release justification to the body of your PR to justify this backport.

Some other things to consider:

• What did we do to ensure that a user that doesn’t know & care about this backport, has no idea that it happened?
• Will this work in a cluster of mixed patch versions? Did we test that?
• If a user upgrades a patch version, uses this feature, and then downgrades, what happens?

[rytaft]

Reviewed 12 of 12 files at r1, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @mgartner)

---

pkg/sql/vars.go line 2417 at r1 (raw file):

			return formatBoolAsPostgresSetting(evalCtx.SessionData().OptimizerAlwaysUseHistograms), nil
		},
		GlobalDefault: globalTrue,

looks like you updated this to true by accident

---

pkg/sql/opt/testutils/opttester/opt_tester.go line 296 at r1 (raw file):

	ot.evalCtx.SessionData().OptimizerUseImprovedSplitDisjunctionForJoins = true
	ot.evalCtx.SessionData().OptimizerAlwaysUseHistograms = true
	// ot.evalCtx.SessionData().OptimizerHoistUncorrelatedEqualitySubqueries = true

debug code

---

pkg/sql/opt/exec/execbuilder/testdata/udf line 99 at r1 (raw file):

----

nit: stray line

[mgartner]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @rytaft)

---

pkg/sql/vars.go line 2417 at r1 (raw file):

Previously, rytaft (Rebecca Taft) wrote…

looks like you updated this to true by accident

Done.

---

pkg/sql/opt/testutils/opttester/opt_tester.go line 296 at r1 (raw file):

Previously, rytaft (Rebecca Taft) wrote…

debug code

Done.

---

pkg/sql/opt/exec/execbuilder/testdata/udf line 99 at r1 (raw file):

Previously, rytaft (Rebecca Taft) wrote…

nit: stray line

Done.

[rytaft]

Reviewed 4 of 4 files at r2, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @mgartner)