Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-17494: [C++] Fix substrait tests linkage on static builds #13939

Merged
merged 5 commits into from
Aug 23, 2022

Conversation

raulcd
Copy link
Member

@raulcd raulcd commented Aug 22, 2022

No description provided.

@raulcd
Copy link
Member Author

raulcd commented Aug 22, 2022

@github-actions crossbow submit test-ubuntu-*-cpp-static

@github-actions
Copy link

@github-actions
Copy link

Revision: 571780b

Submitted crossbow builds: ursacomputing/crossbow @ actions-fc0800e62e

Task Status
test-ubuntu-18.04-cpp-static Github Actions

@@ -58,7 +58,11 @@ foreach(LIB_TARGET ${ARROW_SUBSTRAIT_LIBRARIES})
target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_ENGINE_EXPORTING)
endforeach()

set(ARROW_SUBSTRAIT_TEST_LINK_LIBS ${ARROW_SUBSTRAIT_LINK_lIBS} ${ARROW_TEST_LINK_LIBS})
if(NOT ARROW_SUBSTRAIT_LINK_LIBS)
set(ARROW_SUBSTRAIT_LINK_LIBS substrait)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even-though this fixes the issue I am quite confused on why list(APPEND ARROW_SUBSTRAIT_TEST_LINK_LIBS arrow_substrait_static) is not containing substrait as a dependency to be linked statically as I would have expected. I've been able to reproduce also on UBUNTU=22.04.
@lidavidm what do you think? Any idea on a different approach I could take on this one?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's probably a missing dependency in ThirdpartyToolchain between the generated header and the command that generates it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is enough to reproduce it for me

cmake ../cpp -DCMAKE_BUILD_TYPE=Debug -GNinja -DARROW_BUILD_TESTS=ON -DARROW_BUILD_SHARED=OFF -DARROW_BUILD_STATIC=ON -DARROW_SUBSTRAIT=ON
ninja src/arrow/engine/CMakeFiles/arrow-substrait-substrait-test.dir/substrait/function_test.cc.o

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I don't really get it either. Maybe CMake doesn't export the dependency transitively…? I don't understand CMake well enough to get what's going on here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it has to go under STATIC_INSTALL_INTERFACE_LIBS instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking a look @lidavidm!! I'll keep investigating as I don't think the current fix is good enough and we should find the root cause. Pinging @kou in case he can throw some light on what is going on.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you try the following without your changes?

diff --git a/cpp/cmake_modules/BuildUtils.cmake b/cpp/cmake_modules/BuildUtils.cmake
index 888ca19af5..2b65fcc5f0 100644
--- a/cpp/cmake_modules/BuildUtils.cmake
+++ b/cpp/cmake_modules/BuildUtils.cmake
@@ -399,7 +399,7 @@ function(ADD_ARROW_LIB LIB_NAME)
     endif()
 
     if(ARG_STATIC_LINK_LIBS)
-      target_link_libraries(${LIB_NAME}_static LINK_PRIVATE
+      target_link_libraries(${LIB_NAME}_static PUBLIC
                             "$<BUILD_INTERFACE:${ARG_STATIC_LINK_LIBS}>")
       if(USE_OBJLIB)
         # Ensure that dependencies are built before compilation of objects in

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou this change also solves the issue.

@raulcd
Copy link
Member Author

raulcd commented Aug 23, 2022

@github-actions crossbow submit test-ubuntu-*-cpp-static

@github-actions
Copy link

Revision: 8205c03

Submitted crossbow builds: ursacomputing/crossbow @ actions-b344a34f6c

Task Status
test-ubuntu-18.04-cpp-static Github Actions

@raulcd
Copy link
Member Author

raulcd commented Aug 23, 2022

@github-actions crossbow submit test-ubuntu-*-cpp-static

@raulcd raulcd marked this pull request as ready for review August 23, 2022 12:39
Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@github-actions
Copy link

Revision: a27a9b5

Submitted crossbow builds: ursacomputing/crossbow @ actions-a4566a376b

Task Status
test-ubuntu-18.04-cpp-static Github Actions

@kou kou merged commit 9b23a70 into apache:master Aug 23, 2022
@ursabot
Copy link

ursabot commented Aug 23, 2022

Benchmark runs are scheduled for baseline = 78d586a and contender = 9b23a70. 9b23a70 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.17% ⬆️0.31%] test-mac-arm
[Failed ⬇️0.82% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.28% ⬆️0.07%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 9b23a707 ec2-t3-xlarge-us-east-2
[Failed] 9b23a707 test-mac-arm
[Failed] 9b23a707 ursa-i9-9960x
[Finished] 9b23a707 ursa-thinkcentre-m75q
[Finished] 78d586a4 ec2-t3-xlarge-us-east-2
[Failed] 78d586a4 test-mac-arm
[Failed] 78d586a4 ursa-i9-9960x
[Finished] 78d586a4 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link

ursabot commented Aug 23, 2022

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

anjakefala pushed a commit to anjakefala/arrow that referenced this pull request Aug 31, 2022
zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants