Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

[NSE-254]Fix redundant arrow library issue. #452

Merged
merged 2 commits into from
Aug 6, 2021

Conversation

weiting-chen
Copy link
Collaborator

What changes were proposed in this pull request?

Fix redundant so file pack into jar file.

How was this patch tested?

Already pass TPC-H q1~q5 testing on the local server with below two compile commands.

  1. mvn clean package -Dbuild_arrow=ON -Dbuild_protobuf=ON -Dbuild_jemalloc=OFF -Dcpp_tests=OFF -Dcheck -DskipTests -Pfull-scala-compiler
  2. mvn clean package -Dbuild_arrow=OFF -Dbuild_protobuf=OFF -Dbuild_jemalloc=OFF -Dcpp_tests=OFF -Dcheck -DskipTests -Pfull-scala-compiler

@github-actions
Copy link

github-actions bot commented Aug 5, 2021

#254

@weiting-chen weiting-chen requested a review from zhouyuan August 5, 2021 08:44
@@ -214,45 +214,45 @@ macro(build_arrow STATIC_ARROW)
else()

# Copy Arrow Shared Library to releases directory for package jar
ExternalProject_Add_Step(arrow_ep copy_arrow_binary
#ExternalProject_Add_Step(arrow_ep copy_arrow_binary
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I commit a new one to remove the marked code.

@zhouyuan
Copy link
Collaborator

zhouyuan commented Aug 6, 2021

@weiting-chen
should be able to remove more contents as we are bundled with arrow-data-source, i'll make one more patch for this

@zhouyuan zhouyuan merged commit 91256fd into oap-project:master Aug 6, 2021
zhouyuan pushed a commit to zhouyuan/native-sql-engine that referenced this pull request Aug 8, 2021
* [NSE-254]Fix redundant arrow library issue.

* remote the unneccessary marked code
zhouyuan added a commit that referenced this pull request Aug 9, 2021
* [NSE-254]Fix redundant arrow library issue. (#452)

* [NSE-254]Fix redundant arrow library issue.

* remote the unneccessary marked code

* remove arrow-data-source-common from jar with dependency (#453)

native sql engine is bundled with arrow data source now so
it should be safe to assume those dataset related class will
be 'pre-loaded'

with this fix the jar size is ~97MB(was ~147MB) on my local env

Signed-off-by: Yuan Zhou <[email protected]>

Co-authored-by: Wei-Ting Chen <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants