Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop copying LogicalPlan and Exprs in ReplaceDistinctWithAggregate #10293

Closed
alamb opened this issue Apr 29, 2024 · 3 comments · Fixed by #10460
Closed

Stop copying LogicalPlan and Exprs in ReplaceDistinctWithAggregate #10293

alamb opened this issue Apr 29, 2024 · 3 comments · Fixed by #10460
Assignees

Comments

@alamb
Copy link
Contributor

alamb commented Apr 29, 2024

Is your feature request related to a problem or challenge?

Part of #9637

As part of making the planner faster, we are updating the optimizer passes to avoid copying LogicalPlan and Expr (see #9637)

Describe the solution you'd like

I would like to reduce the amount of copying in this pass (even though it doesn't appear in current profiling)

Describe alternatives you've considered

Apply the model from @Lordworms in #10166 to this pass
2. Update OptimizerRule::supports_rewrite` to return true

  1. Update OptimizerRule to use rewrite
  2. Update the pass itself to not copy the LogicalPlan (ideally using the TreeNode API) - it is implemented for LogicalPlan (API) and Expr (API)

Other examples: #10218

Additional context

@ClSlaid
Copy link
Contributor

ClSlaid commented May 11, 2024

May I take a look? Merci.

@alamb
Copy link
Contributor Author

alamb commented May 11, 2024

of course thank you @ClSlaid -- just be aware these PRs have tended to be tricky. I recommend doing it incrementally if possible. Let me know if you need some help 🙏

@ClSlaid
Copy link
Contributor

ClSlaid commented May 11, 2024

hi, @alamb, I'd like to know if those two expressions are semantically identical:

                let aggr_expr = vec![Expr::AggregateFunction(AggregateFunction::new(
                    AggregateFunctionFunc::FirstValue,
                    select_expr,
                    false,
                    None,
                    sort_expr.clone(),
                    None,
                ))];
                let aggr_expr = select_expr.into_iter().map(|e| {
                    Expr::AggregateFunction(AggregateFunction::new(
                        AggregateFunctionFunc::FirstValue,
                        vec![e],
                        false,
                        None,
                        sort_expr.clone(),
                        None,
                    ))
                }).collect::<Vec<_>>();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants