Skip to content

Commit

Permalink
apacheGH-39265: [Java] Make it run well with the netty newest version…
Browse files Browse the repository at this point in the history
… 4.1.104 (apache#39266)

When I used `netty arrow memory 14.0.1` and `netty 4.1.104.Final` in Spark, the following error occurred,
After pr: netty/netty#13613, `PoolArena` no longer extends `SizeClasses`, but instead uses it as one of its fields, as follows:
<img width="1051" alt="image" src="https://github.com/apache/arrow/assets/15246973/6112757b-b2b7-42aa-b4c1-6ab473b91a09">
in order to ensure that `netty arrow memory 14.0.1` works well with `netty 4.1.104.Final` version, I suggest making similar modifications here.
1.Compilation errors are as follows:
https://github.com/panbingkun/spark/actions/runs/7237466030/job/19717162391
<img width="1005" alt="image" src="https://github.com/apache/arrow/assets/15246973/98edb6a1-f0e6-4d4e-b568-fbdbffe612f0">

2.Some bugs have been fixed in `netty 4.1.104.Final` as follows:
<img width="862" alt="image" src="https://github.com/apache/arrow/assets/15246973/12354a1e-cddd-4ab8-b168-e92712d84cea">
<img width="861" alt="image" src="https://github.com/apache/arrow/assets/15246973/bd7d27e1-3953-451c-8c9b-24ecb0d61efd">

4.1.104.Final release note: https://netty.io/news/2023/12/15/4-1-104-Final.html
4.1.103.Final release note: https://netty.io/news/2023/12/13/4-1-103-Final.html
4.1.101.Final release note: https://netty.io/news/2023/11/09/4-1-101-Final.html

Java
* Closes: apache#39265

Authored-by: panbingkun <[email protected]>
Signed-off-by: David Li <[email protected]>
  • Loading branch information
panbingkun authored and lriggs committed Sep 6, 2024
1 parent 46b3e77 commit b8aea7f
Showing 1 changed file with 4 additions and 14 deletions.
18 changes: 4 additions & 14 deletions java/gandiva/src/main/cpp/jni_common.cc
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,6 @@ DataTypePtr ProtoTypeToList(const types::ExtGandivaType& ext_type) {
DataTypePtr SimpleProtoTypeToDataType(const types::GandivaType& gandiva_type) {
switch (gandiva_type) {
case types::NONE:
>>>>>>> a225426e1 (DX-64328 Array types for Gandiva (#58))
return arrow::null();
case gandiva::types::BOOL:
return arrow::boolean();
Expand Down Expand Up @@ -295,9 +294,9 @@ DataTypePtr SimpleProtoTypeToDataType(const types::GandivaType& gandiva_type) {
return arrow::date32();
case gandiva::types::DATE64:
return arrow::date64();
<<<<<<< HEAD
case gandiva::types::DECIMAL:
=======
// TODO: error handling
return arrow::decimal(ext_type.precision(), ext_type.scale());
default:
std::cerr << "Unknown data type: " << gandiva_type << "\n";
return nullptr;
Expand All @@ -309,7 +308,6 @@ DataTypePtr SimpleProtoTypeToDataType(const types::GandivaType& gandiva_type) {
DataTypePtr ProtoTypeToDataType(const types::ExtGandivaType& ext_type) {
switch (ext_type.type()) {
case types::DECIMAL:
>>>>>>> a225426e1 (DX-64328 Array types for Gandiva (#58))
// TODO: error handling
return arrow::decimal(ext_type.precision(), ext_type.scale());
case gandiva::types::TIME32:
Expand All @@ -320,21 +318,13 @@ DataTypePtr ProtoTypeToDataType(const types::ExtGandivaType& ext_type) {
return ProtoTypeToTimestamp(ext_type);
case gandiva::types::INTERVAL:
return ProtoTypeToInterval(ext_type);
<<<<<<< HEAD
case gandiva::types::FIXED_SIZE_BINARY:
case gandiva::types::LIST:
return ProtoTypeToList(ext_type);
case gandiva::types::FIXED_SIZE_BINARY:
case gandiva::types::STRUCT:
case gandiva::types::UNION:
case gandiva::types::DICTIONARY:
case gandiva::types::MAP:
=======
case types::LIST:
return ProtoTypeToList(ext_type);
case types::FIXED_SIZE_BINARY:
case types::UNION:
case types::DICTIONARY:
case types::MAP:
>>>>>>> a225426e1 (DX-64328 Array types for Gandiva (#58))
std::cerr << "Unhandled data type: " << ext_type.type() << "\n";
return nullptr;
default:
Expand Down

0 comments on commit b8aea7f

Please sign in to comment.