Move datafusion_array_function
specific rewrite rules like to datafusion_functions_array
crate
#9519
Labels
enhancement
New feature or request
Is your feature request related to a problem or challenge?
As we port over functions out of the
datafusion-physical-expr
crate, we have discovered several parts ofExprSimplif
that rely on semantics of the array functions themselvesFor example, the rewrite from
array1 @> array2
to array_has_all(array1, array2)in
rewrite_array_has_all_operator_to_func`([source link](https://github.com/apache/arrow-datafusion/blob/5537572820977b38719e2253f601a159deef5bc6/datafusion/optimizer/src/analyzer/rewrite_expr.rs#L137
As @jayzhan22 noted on #9496 #9496 (comment), this rewrite is not correct if users supply their own implementation of
array_has_all
as that implementation may not be equal to DataFusion'sarray_has_all
function. Thus the rule only makes sense when using the datafusion implementationDescribe the solution you'd like
Move the array_function specific rules such as
rewrite_array_has_all_operator_to_func
to thedataufusion_array_functions
Describe alternatives you've considered
Analyzer
pass (defined indatafusion_functions_array
) with the desired rewritedatafusion_function_array
specific rewrite when the array functions indatafusion_array_function
are registered (somewhere near hereNote you can register an analyzer rule via
SessionState::add_analyzer_rule
Additional context
No response
The text was updated successfully, but these errors were encountered: