-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve aggregation code readability #12335
Comments
I think it is a good chance to add documentation that helps yourself and others, or refactor the code to improve the codebase. |
Yeah, I will try. I'm not familiar with the whole code yet, I'll keep reading. |
This would be a valuable improvement. Now the execution behavior is determined by |
I think at least we should separate partial and the teminals(Final, Single...). |
I was also reading the aggregation code and came up with one possible simplification. What do you think about the idea of a change like this #12517 I think it helps simplify the logic as it removes the |
I guess it may be a bit confused? Because an aggr operator wraps another aggr operator. |
Yeah, I guess it a bit of a double edge sword that it just moves the complexity elsewhere. But I could think well together with your suggestion of
Because without doing something like this the partial would still need all the logic for the terminal. Or at least my understand is that the stream merging is basically just performing a terminal aggregation. |
Maybe we can refactor the codes thoroughly after #11943 (which can obviously improve aggr performance) is merged. I think the
And some paths only belong to
|
Is your feature request related to a problem or challenge?
I'm reading aggregation code recently, found it's hard to read due to different
AggregateMode
, these modes are mixed up in the code.Describe the solution you'd like
Might split stream based on
AggregateMode
?Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: