-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Epic] Unify AggregateFunction
Interface (remove built in list of AggregateFunction
s), improve the system
#8708
Comments
AggregateFunction
Interface (remove built in list of AggregateFunction
s)AggregateFunction
Interface (remove built in list of AggregateFunction
s), improve the system
I will work on FirstValue UDF together with #9249 |
Can we introduce |
Thank you @jayzhan211 -- I am traveling this week so I am very behind on reviews. I will try and respond later this week |
I leave the comment so I won't forget. |
This comment was marked as resolved.
This comment was marked as resolved.
* Move Regr_* functions to use UDAF Closes apache#10883 and is part of apache#8708 * Format and regen * tweak error check --------- Co-authored-by: Andrew Lamb <[email protected]>
Is your feature request related to a problem or challenge?
For many of the same reasons as listed on #8045, having two types of aggregate functions ("built in" -- datafusion::physical_plan::aggregates::AggregateFunction) and AggregateUDF is problematic for two reasons:
GroupsAccumulator
interface)The second also ends up causing pushback on adding new aggregates like
ARRAY_SUM
in #8325 and geospatial support #7859.Describe the solution you'd like
I propose moving DataFusion to only use
AggregateUDF
s and remove the built in list of AggregateFunctions for the same reasons as #8045We will keep the existing
AggregateUDF
interface as much as possible, while also potentially providing an easier way to define them.New AggregateUDF is in
functions-aggregate
crateOld Aggregate functions are in
datafusion/physical-expr/src/aggregate
Describe alternatives you've considered
Additional context
Proposed implementation steps:
name
for aggregate function #10695AggregateUDF
#8710GroupsAccumulator
accumulator for User Defined Functions #8793AggregateExpr
,PhysicalExpr
andPhysicalSortExpr
to physical-expr-core #9926AggregateUDFImpl::simplfy()
API #9526Covariance
(Sample)covar
/covar_samp
to be a User Defined Aggregate Function #10372Count
tofunctions-aggregate
, update MSRV to rust 1.75 #10484functions-aggregate
and Introduce Numeric signature #10644AggregateUDFImpl::accumulator
signature, support ORDER BY for UDAFs #9874 Convert first, last aggregate function to UDAF #10648Variance Sample
to UDAF #10667stddev
to udaf #10827approx_median
to UDAF #10838ApproxPercentileCont
and `ApproxPercentileContWithWeight
to UDAF #10870Regr
to UDAF #10883Correlation
to UDAF #10884BitAnd
,BitOr
,BitXor
to UDAF #10907Average
to UDAF #10942Min
andMax
to UDAF #10943StringAgg
to UDAF #10945ArrayAgg
to UDAF #10999BoolAndOr
to UDAF #11008Min
andMax
to UDAF #10943Min
/Max
built inAggregateFunctions
#11151Move rust test to sqllogictest if possible #10384
Good first issue list
Grouping
to UDAF #10906Pending
Feel free to file an issue if you are interested in working on any of the above in the pending list.
The text was updated successfully, but these errors were encountered: