Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Badcase for collect_list #398

Closed
lgbo-ustc opened this issue Apr 4, 2023 · 1 comment · Fixed by apache/incubator-gluten#1301 or #404
Closed

Badcase for collect_list #398

lgbo-ustc opened this issue Apr 4, 2023 · 1 comment · Fixed by apache/incubator-gluten#1301 or #404

Comments

@lgbo-ustc
Copy link

lgbo-ustc commented Apr 4, 2023

(you don't have to strictly follow this form)

Describe the unexpected behaviour

select n_regionkey, collect_list(if(n_regionkey=0, n_name, null)) as t from nation_nullable group by n_regionkey

The above query will throw an exception

java.lang.RuntimeException: Cannot convert NULL to a non-nullable type: while converting source column `collect_list#1#Partial#collect_list` to destination column `collect_list#1#Partial#collect_list`
0. ./build1/../contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x18fd05da in /data3/liangjiabiao/github/KyClickHouse/build1/utils/local-engine/libch.so
1. ./build1/../src/Common/Exception.cpp:91: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0xe481875 in /data3/liangjiabiao/github/KyClickHouse/build1/utils/local-engine/libch.so
2. ./build1/../contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const (&) [43], void>(int, char const (&) [43]) @ 0xbb8abd6 in /data3/liangjiabiao/github/KyClickHouse/build1/utils/local-engine/libch.so
3. ./build1/../src/Functions/FunctionsConversion.h:3678: DB::FunctionCast<DB::CastInternalName>::prepareUnpackDictionaries(std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::IDataType const> const&) const @ 0xbc4b4fc in /data3/liangjiabiao/github/KyClickHouse/build1/utils/local-engine/libch.so
4. ./build1/../src/Functions/FunctionsConversion.h:2780: DB::FunctionCast<DB::CastInternalName>::prepare(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0xbc4a986 in /data3/liangjiabiao/github/KyClickHouse/build1/utils/local-engine/libch.so
5. ./build1/../contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::ActionsDAG::addFunctionImpl(std::__1::shared_ptr<DB::IFunctionBase const> const&, std::__1::vector<DB::ActionsDAG::Node const*, std::__1::allocator<DB::ActionsDAG::Node const*> >, std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool) @ 0x143e8c77 in /data3/liangjiabiao/github/KyClickHouse/build1/utils/local-engine/libch.so

Since array type is nullable in gluten, the result from groupArray could be null now. But spark also require the result from collect_list non-nullable.

@lgbo-ustc
Copy link
Author

WIP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant