-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial changes to support using udaf min/max for statistics and opti… #11696
Conversation
d777951
to
0addab0
Compare
datafusion/expr/src/udaf.rs
Outdated
/// None in all other cases, used in certain optimizations or | ||
/// or aggregate | ||
/// | ||
pub fn get_minmax_desc(&self) -> Option<bool> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can I suggest using an enum
like enum MinMax { Min, Max }
instead of a bool
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is the code that we need the boolean, maybe name it with is_descending: Option<bool>
?
let (field, desc) = aggr.get_minmax_desc()?;
if desc != order.options.descending {
return None;
}
And, we can get fields separately
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed with eb0ebe0
(#11696) although I am not sure I understood what "is descending" mean for a function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is the same idea like descending in SortOptions
pub struct SortOptions {
/// Whether to sort in descending order
pub descending: bool,
Would is_sorted_descending: Option<bool>
be much more better?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let field = aggr.field()
let is_sorted_descending = aggr.is_sorted_descending()
And, I think getting field and descending separately is better than get_minmax_desc
, so if we can get field
only if we don't care about descending
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. The goal of this PR was to simplify #10943 by implementing here the necessary method for udaf, do you think I should proceed in the refactoring now or can we do as a separate improvement?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not necessary
Thanks @edmondop |
This PR tries to extract some work necessary for #10943. Min and Max are special functions that are used for statistics and optimizations, and they need a special treatment.