opt: consider limit hint when costing distinct-on expressions #122442
Conversation
This commit adds coster tests for distinct-on expressions. There were no such tests before. Release note: None
|
This change is |
There was a problem hiding this comment.
Reviewed 1 of 1 files at r1, 3 of 3 files at r2, 12 of 12 files at r3, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @michae2)
mgartner
force-pushed
the
fuzzy-search-distinct
branch
from
7e2bada
to
8332dc7
Compare
Limit hints are now considered when costing DistinctOn expressions, similarly to streaming GroupBy expressions. This matches the behavior of the execution logic, where a DistinctOn emits a row with distinct grouping columns as soon as it is read from its input. It does not need to read all of its input to begin emitting rows. Release note (performance improvement): The optimizer now costs distinct-on operators more accurately. It may produce more efficient query plans in some cases.
The `optimizer_use_improved_distinct_on_limit_hint_costing` session setting has been added which enables the improved costing of DistinctOn expressions with limit hints added in the previous commit. It is enabled by default. Release note: None
mgartner
force-pushed
the
fuzzy-search-distinct
branch
from
8332dc7
to
af62526
Compare
| outputRowCount = math.Min(outputRowCount, required.LimitHint) | ||
| } else if grouping.Op() == opt.DistinctOnOp && | ||
| c.evalCtx.SessionData().OptimizerUseImprovedDistinctOnLimitHintCosting { | ||
| inputRowCount = distinctOnLimitHint(outputRowCount, required.LimitHint) |
There was a problem hiding this comment.
Note: We might need to use streamingGroupByInputLimitHint for streaming distinct-on expressions, to make the plan changes go away.
|
There are a few intriguing plan changes, mostly from streaming distinct-ons to hash distinct-ons. @DrewKimball and I spent some time looking at this and we aren't super concerned with the changes—they should have very little impact on performance, if any. #122644 tracks some potential follow-up work. |
mgartner
added
backport-23.1.x
|
bors r+ |
![blathers[crl]](https://avatars.githubusercontent.com/in/59700?s=40&u=373d9101a12edc3847d2fb6ae78e3adcfb4ca41e&v=4){kind=avatar}
|
Encountered an error creating backports. Some common things that can go wrong:
You might need to create your backport manually using the backport tool. error creating merge commit from af62526 to blathers/backport-release-23.1-122442: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 23.1.x failed. See errors above. error creating merge commit from af62526 to blathers/backport-release-23.2-122442: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 23.2.x failed. See errors above. error creating merge commit from af62526 to blathers/backport-release-24.1-122442: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 24.1.x failed. See errors above. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
opt: add coster tests for disctinct-on expressions
This commit adds coster tests for distinct-on expressions. There were no
such tests before.
Release note: None
opt: consider limit hint when costing distinct-on expressions
Limit hints are now considered when costing DistinctOn expressions,
similarly to streaming GroupBy expressions. This matches the behavior of
the execution logic, where a DistinctOn emits a row with distinct
grouping columns as soon as it is read from its input. It does not need
to read all of its input to begin emitting rows.
Release note (performance improvement): The optimizer now costs
distinct-on operators more accurately. It may produce more efficient
query plans in some cases.
Epic: CRDB-37714
opt: add optimizer_use_improved_distinct_on_limit_hint_costing
The
optimizer_use_improved_distinct_on_limit_hint_costingsessionsetting has been added which enables the improved costing of DistinctOn
expressions with limit hints added in the previous commit. It is enabled
by default.
Release note: None