Support FixedSizeList in array methods

[izveigor]

Is your feature request related to a problem or challenge?

Follow on to #6384

Data can come from different sources (such as apache-avro, parquet ...) where the list data type is supported, which can become FixedSizeList in arrow-datafusion. So as not to cause difficulties regarding the casting between FixedSizeList and List, I suggest to support FixedSizeList in all array functions.

Describe the solution you'd like

Pre casting FixedSizeList to List before calling array functions or native support for that data type.

Describe alternatives you've considered

No response

Additional context

Simillar Issues:
#6119
#6075

[jayzhan211]

We should do coercion in datafusion/optimizer/src/analyzer/type_coercion.rs, right?

[alamb]

We should do coercion in datafusion/optimizer/src/analyzer/type_coercion.rs, right?

Yes, that is the preferred location (as it happens before any optimization passes) -- the reason is that type coercion can change the semantics of the plan, but the optimizer passes should not

https://docs.rs/datafusion/latest/datafusion/index.html#planning has more information

[jayzhan211]

My first thought is to add casting from FixedSizeList to List in coerce_arguments_for_signature.

However, I fail to construct a test.
I don't know how to create a test case with DataType::FixedSizeList in sqllogictests array.slt. make_array([1,2]) gives us ScalarValue::List which converted to DataType::List, and since the args in ScalarFuncion for TypeCoercionRewriter is Vec<Expr>, I could only construct Expr::Literal(ScalarValue::List), no idea to construct FixedSizeList from Expr and ScalarValue in a unit test, any idea about this?
https://github.com/apache/arrow-datafusion/blob/84e49771b7403b3d313d8493b61d2d58dcdd7514/datafusion/optimizer/src/analyzer/type_coercion.rs#L383

[alamb]

Since FixedSizeList is a different Arrow type, we probably need a new ScalarValue variant for it. Maybe ScalarValue::FixedSizedList 🤔

https://docs.rs/datafusion/latest/datafusion/scalar/enum.ScalarValue.html#variant.List

[alamb]

I wonder if a reasonable approach would be to start with the code to cast to/from List and FixedSizedList -- that way there would at least be some path for working with data that came in as FixedSizedList

So like

SELECT 
  arrow_cast(my_column, 'List(Utf8)') -- would call the `cast` kernel to convert `my_column` to `LIstArray`
FROM
  my_parquet_table_with_fixed_size_lists

[jayzhan211]

I think we have support FixedSizeList already. Thanks for @Weijun-H