feature: Add a WindowUDFImpl::simplify() API

[guojidan]

Which issue does this PR close?

Closes #9527 .

Rationale for this change

allow user define simply rule

What changes are included in this PR?

add WindowUDFImpl::simplfy() API

Are these changes tested?

Are there any user-facing changes?

[guojidan]

hi @jayzhan211 , maybe you can review this PR 😄

[jayzhan211]

@guojidan I think we need a simple example. Something like simplifying nth_value() to first_value() if n is 1

[guojidan]

@guojidan I think we need a simple example

separate create a new UDWF example file?

[jayzhan211]

We have example of cast to i64 for scalar function in https://github.com/apache/arrow-datafusion/blob/cc1db8a2043c73bda7adec309b42c08d88defab8/datafusion/core/tests/user_defined/user_defined_scalar_functions.rs#L524

You need to write an example in datafusion/core/tests/user_defined/user_defined_window_functions.rs and shows that the simply() works and make sense

[alamb]

I agree with @jayzhan211 that an example would be great ❤️

[alamb]

Thank you for the contribution @guojidan

[alamb]

This might be confusing as now this example UDWF will never call the actual implementation.

Perhaps we can make a second UDWF and use it as an example of wrapping an existing UDWF 🤔

[alamb]

Ah this is interesting. I was trying to figure out how we would update this signature to avoid a copy (for example, a rewrite would have to copy the order_by exprs 🤔

I can't think of anything at the moment

[milenkovicm]

it looks quite hard to come up with better backward compatible solution

[milenkovicm]

It would be great if we would not need do to decompose and then compose function again, something like:

          Expr::AggregateFunction(AggregateFunction { 
              func_def:AggregateFunctionDefinition::UDF(ref udaf), 
              ref args, 
              ref distinct, 
              ref filter, 
              ref order_by, 
              ref null_treatment }) => {
              match udaf.simplify(args, distinct, &filter, &order_by, &null_treatment)? {
                  ExprSimplifyResult::Simplified(simplified) => Transformed::yes(simplified),
                  ExprSimplifyResult::Original(_) => Transformed::no(expr),
              }
          }

clone of args could be avoided if simplify returns empty vector ... but that breaks semantics of the api

[comphead]

Thats probably better to use builder pattern, if the mandatory and optional input params are different

[guojidan]

I don't quite understand what to do. Can you further explain 😄

[alamb]

Marking as draft as I think this PR is no longer waiting on feedback. Please mark it as ready for review when it is ready for another look

[milenkovicm]

I guess at some point we'd need to change UDF::simplify to align interfaces

[jayzhan211]

remind to cleanup this

[jayzhan211]

👍

[alamb]

Thanks @guojidan and @jayzhan211 for the review

I think there are some cleanups (liek https://github.com/apache/datafusion/pull/9906/files#r1618798950 from @jayzhan211 ) that we should do but we can do them as follow on PRs too.

Thank you