Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R] MacOS autobrew jobs fail to build Google Cloud Storage formula (bad curl headers?) #36686

Closed
paleolimbot opened this issue Jul 14, 2023 · 6 comments · Fixed by #36706
Closed
Assignees
Milestone

Comments

@paleolimbot
Copy link
Member

Describe the bug, including details regarding any error messages, version, and platform.

The homebrew-r-autobrew and r-binary-packages jobs are failing for at least two reasons. The first is an OpenSSL issue related to 1.1/3 mismatch (#36456); the second is preventing a verification that #36551 fixes the OpenSSL issue.

The only hypothesis I have is that the MacOS runner is actually an M1 emulating x86 via rosetta and perhaps the clang links are getting mixed up when autobrew is invoked?

An example failure from r-binary-packages ( https://github.com/ursacomputing/crossbow/actions/runs/5540883095/jobs/10113554437 ) (failure from homebrew-r-autobrew is currently the same)

In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/__hash_table:18:
/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/cmath:321:9: error: no member named 'signbit' in the global namespace
using ::signbit;
      ~~^
/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/internal/curl_wrappers.h:18:
In file included from /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/options.h:26:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/unordered_map:411:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/cmath:322:9: error: /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/__hash_table:18:
no member named 'fpclassify' in the global namespace/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/cmath:321:9: error: 
using ::fpclassify;
      ~~^
no member named 'signbit' in the global namespace
using ::signbit;
      ~~^
/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/cmath:323:9: error: no member named 'isfinite' in the global namespace; did you mean 'finite'?
using ::isfinite;
      ~~^
/Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/math.h:752:12: note: 'finite' declared here
extern int finite(double)
           ^
/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/cmath:322:9: error: no member named 'fpclassify' in the global namespace
using ::fpclassify;
      ~~^

Component(s)

R

@paleolimbot
Copy link
Member Author

Hmm...this may be the same as the OpenSSL issue after all: the failing command comes from compiling Google Cloud Platform curl_handle.cc ( https://github.com/ursacomputing/crossbow/actions/runs/5540883095/jobs/10113554437#step:7:5792 ).

 cd /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep-build/google/cloud && /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow-static/Library/Homebrew/shims/mac/super/clang++ -DJSON_DIAGNOSTICS=0 -DJSON_USE_IMPLICIT_CONVERSIONS=1 -I/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep -isystem /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/absl_ep-install/include -isystem /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep-build -isystem /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include -isystem /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/nlohmann_json_ep-install/include -Qunused-arguments -fcolor-diagnostics -fPIC -O3 -DNDEBUG -O2 -std=c++17 -isysroot /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -mmacosx-version-min=11.7 -Wall -Wextra -Wconversion -Wno-sign-conversion -MD -MT google/cloud/CMakeFiles/google_cloud_cpp_rest_internal.dir/internal/curl_handle.cc.o -MF CMakeFiles/google_cloud_cpp_rest_internal.dir/internal/curl_handle.cc.o.d -o CMakeFiles/google_cloud_cpp_rest_internal.dir/internal/curl_handle.cc.o -c /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/internal/curl_handle.cc

There seem to be some successful calls to the compiler earlier on so it's not that:

[ 96%] Building CXX object absl/flags/CMakeFiles/flags_usage_internal.dir/internal/usage.cc.o
cd /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/absl_ep-prefix/src/absl_ep-build/absl/flags && /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow-static/Library/Homebrew/shims/mac/super/clang++  -I/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/absl_ep-prefix/src/absl_ep -Qunused-arguments -fcolor-diagnostics -fPIC -O3 -DNDEBUG -O2 -std=gnu++17 -isysroot /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -mmacosx-version-min=11.7 -Wall -Wextra -Wcast-qual -Wconversion -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wfor-loop-analysis -Wformat-security -Wgnu-redeclared-enum -Winfinite-recursion -Winvalid-constexpr -Wliteral-conversion -Wmissing-declarations -Woverlength-strings -Wpointer-arith -Wself-assign -Wshadow-all -Wstring-conversion -Wtautological-overlap-compare -Wundef -Wuninitialized -Wunreachable-code -Wunused-comparison -Wunused-local-typedefs -Wunused-result -Wvla -Wwrite-strings -Wno-float-conversion -Wno-implicit-float-conversion -Wno-implicit-int-float-conversion -Wno-implicit-int-conversion -Wno-shorten-64-to-32 -Wno-sign-conversion -Wno-unknown-Note-option -DNOMINMAX -MD -MT absl/flags/CMakeFiles/flags_usage_internal.dir/internal/usage.cc.o -MF CMakeFiles/flags_usage_internal.dir/internal/usage.cc.o.d -o

@paleolimbot paleolimbot changed the title [R] MacOS autobrew jobs fail to configure (crash or fail citing ABI mismatch) [R] MacOS autobrew jobs fail to build Google Cloud Storage formula (bad curl headers?) Jul 14, 2023
@paleolimbot
Copy link
Member Author

Would a reasonable hypothesis be that we are using system curl but autobrew openssl and GCS makes some assumption about how that interaction works that other components do not?

@paleolimbot
Copy link
Member Author

Hmm...trying homebrew curl gets us a little farther in the build process but still results in a similar error.

Something similar is described here: https://stackoverflow.com/questions/58628377/catalina-c-using-cmath-headers-yield-error-no-member-named-signbit-in-th

...something about CMAKE_OSX_SYSROOT.

@kou
Copy link
Member

kou commented Jul 15, 2023

It seems that MacOSX${VERSION}.sdks are mixed:

  • -isystem .../MacOSX12.1.sdk
  • .../MacOSX11.sdk/usr/include/c++/v1/unordered_map

https://github.com/ursacomputing/crossbow/actions/runs/5540883095/jobs/10113554437#step:7:5793

cd /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep-build/google/cloud && /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/build-apache-arrow-static/Library/Homebrew/shims/mac/super/clang++ -DJSON_DIAGNOSTICS=0 -DJSON_USE_IMPLICIT_CONVERSIONS=1 -I/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep -isystem /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/absl_ep-install/include -isystem /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep-build -isystem /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include -isystem /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/nlohmann_json_ep-install/include -Qunused-arguments -fcolor-diagnostics -fPIC -O3 -DNDEBUG -O2 -std=c++17 -isysroot /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -mmacosx-version-min=11.7 -Wall -Wextra -Wconversion -Wno-sign-conversion -MD -MT google/cloud/CMakeFiles/google_cloud_cpp_rest_internal.dir/internal/binary_data_as_debug_string.cc.o -MF CMakeFiles/google_cloud_cpp_rest_internal.dir/internal/binary_data_as_debug_string.cc.o.d -o CMakeFiles/google_cloud_cpp_rest_internal.dir/internal/binary_data_as_debug_string.cc.o -c /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/internal/binary_data_as_debug_string.cc
In file included from /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/internal/curl_handle_factory.cc:15:
In file included from /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/internal/curl_handle_factory.h:18:
In file included from In file included from /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/internal/curl_handle.cc:15:
In file included from /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/internal/curl_handle.h:18:
In file included from /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/internal/curl_wrappers.h:18:
In file included from /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/hbtmp/apache-arrow-static-20230713-13809-1f91jjk/build/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep/google/cloud/options.h:26:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/unordered_map:411:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/__hash_table:18:
/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/c++/v1/cmath:321:9: error: no member named 'signbit' in the global namespace
using ::signbit;
      ~~^

paleolimbot pushed a commit that referenced this issue Jul 16, 2023
### Rationale for this change

If we use different macOS SDK in Apache Arrow C++ and bundled projects, it will cause some problems such as a build error.

### What changes are included in this PR?

Pass `CMAKE_OSX_SYSROOT` explicitly to external projects.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* Closes: #36686

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Dewey Dunnington <[email protected]>
@paleolimbot paleolimbot added this to the 14.0.0 milestone Jul 16, 2023
@paleolimbot
Copy link
Member Author

@raulcd I think this is an important one to pull into 13.0.0 if possible!

@raulcd
Copy link
Member

raulcd commented Jul 17, 2023

oh! I see! Thanks @paleolimbot , I'll wait for some jobs to finish and I'll cherry pick again.

@raulcd raulcd modified the milestones: 14.0.0, 13.0.0 Jul 17, 2023
raulcd pushed a commit that referenced this issue Jul 17, 2023
### Rationale for this change

If we use different macOS SDK in Apache Arrow C++ and bundled projects, it will cause some problems such as a build error.

### What changes are included in this PR?

Pass `CMAKE_OSX_SYSROOT` explicitly to external projects.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* Closes: #36686

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Dewey Dunnington <[email protected]>
chelseajonesr pushed a commit to chelseajonesr/arrow that referenced this issue Jul 20, 2023
…ache#36706)

### Rationale for this change

If we use different macOS SDK in Apache Arrow C++ and bundled projects, it will cause some problems such as a build error.

### What changes are included in this PR?

Pass `CMAKE_OSX_SYSROOT` explicitly to external projects.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* Closes: apache#36686

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Dewey Dunnington <[email protected]>
R-JunmingChen pushed a commit to R-JunmingChen/arrow that referenced this issue Aug 20, 2023
…ache#36706)

### Rationale for this change

If we use different macOS SDK in Apache Arrow C++ and bundled projects, it will cause some problems such as a build error.

### What changes are included in this PR?

Pass `CMAKE_OSX_SYSROOT` explicitly to external projects.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* Closes: apache#36686

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Dewey Dunnington <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants