-
Notifications
You must be signed in to change notification settings - Fork 5.4k
Troubleshooting known build issues
Observed issue
[80/85] Linking CXX executable bin/fizz-bogoshim
FAILED: bin/fizz-bogoshim
: && /Library/Developer/CommandLineTools/usr/bin/c++ -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.0.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/BogoShim.dir/test/BogoShim.cpp.o -o bin/fizz-bogoshim -Wl,-rpath,/opt/homebrew/lib lib/libfizz.a /opt/homebrew/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib /usr/local/lib/libfolly.a /usr/local/lib/libfmt.a /opt/homebrew/lib/libboost_context-mt.dylib /opt/homebrew/lib/libboost_filesystem-mt.dylib /opt/homebrew/lib/libboost_atomic-mt.dylib /opt/homebrew/lib/libboost_program_options-mt.dylib /opt/homebrew/lib/libboost_regex-mt.dylib /opt/homebrew/lib/libboost_system-mt.dylib /opt/homebrew/lib/libboost_thread-mt.dylib /opt/homebrew/lib/libevent.dylib /Library/Developer/CommandLineTools/SDKs/MacOSX13.0.sdk/usr/lib/libz.tbd /Library/Developer/CommandLineTools/SDKs/MacOSX13.0.sdk/usr/lib/libbz2.tbd /opt/homebrew/lib/liblz4.dylib /opt/homebrew/lib/libsnappy.dylib /opt/homebrew/lib/libsodium.dylib -lc++abi /opt/homebrew/opt/[email protected]/lib/libssl.dylib /opt/homebrew/opt/[email protected]/lib/libcrypto.dylib /Library/Developer/CommandLineTools/SDKs/MacOSX13.0.sdk/usr/lib/libz.tbd /opt/homebrew/lib/libzstd.dylib /opt/homebrew/lib/libglog.dylib /opt/homebrew/lib/libgflags.2.2.2.dylib /usr/local/lib/libdouble-conversion.a && :
Undefined symbols for architecture arm64:
"_EVP_PKEY_get_bits", referenced from:
folly::AsyncSSLSocket::getSSLCertSize() const in libfolly.a(AsyncSSLSocket.cpp.o)
"_SSL_get1_peer_certificate", referenced from:
folly::AsyncSSLSocket::getPeerCertificate() const in libfolly.a(AsyncSSLSocket.cpp.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Explanation
Homebrew has installed two versions of Openssl, version 1.1 and 3 (or later) due to package dependencies. Version 1.1 is required to be used for Prestissimo. Version 3 is now found in the system path. While the build script will link the library from version 1.1 it includes the headers from version 3 which changes the symbols that need to be provided by the library. Effectively, the issue is caused by a version mismatch of the header and library.
Check the installation by looking at what is installed using
% brew list | grep openssl
[email protected]
openssl@3
Workaround
Until the problem has been addressed in the CMake make files the following options exist:
- Uninstall openssl@3. This might cause issues with other programs installed that depend on this version.
- Prior to building Prestissimo unlink version 3 and link version 1.1. Then build, and once complete or version 3 is needed for other activities to be in the system path again, unlink version 1.1 and link version 3 again. This can be accomplished by the following commands:
brew unlink openssl@3
brew link [email protected]
and if needed
brew unlink [email protected]
brew link openssl@3
Make sure you have the following line in your /etc/apt/sources.list:
deb http://archive.ubuntu.com/ubuntu/ jammy universe
Problem
During the build and installation of the dependencies the install fails with permission denied errors.
The usual cause is that the user running the script does not have the necessary permissions to install to the target directory. by default, the dependencies are installed to /usr/local which may not be accessible to the user.
Example error output:
+ cmake --install _build
-- Install configuration: ""
-- Installing: /usr/local/lib/libfolly.a
CMake Error at _build/cmake_install.cmake:41 (file):
file INSTALL cannot copy file
"/Users/foobar/Desktop/repos/prestodb/prestissimo-deps/folly/_build/libfolly.a"
to "/usr/local/lib/libfolly.a": Permission denied.
Solution
Review the README of Velox. Prestissimo sources the Velox setup scripts and both require the same basic configuration.
The recommended solution is to set the INSTALL_PREFIX environment variable as target for the dependencies.
Refer to https://github.com/facebookincubator/velox/blob/3bf1a74042e83981630d24c1eaee190d1f31aac2/README.md#setting-up-dependencies.
Example error output:
clang: error: no such file or directory: '/Users/foobar/Desktop/repos/prestodb/presto/presto-native-execution/_build/release/velox/velox/expression/type_calculation/TypeCalculation.yy.cc'
clang: error: no input files
Explanation
This error occurs when certain C++ files that are generated by bison or m4 cannot be found during compilation. These are files that are generated when the Velox submodule compiles.
For example, the TypeParserScanner, TypeCalculationScanner, and SignatureParserScanner. An error could be that TypeParser.cpp is not found which in turn means that the generation of the file failed.
Solution For MacOS users, it is best to have the homebrew bison and m4 utility in the path prior to getting the system version of these utilities.
It is recommended to add the following to the PATH environment variable in the profile (zshrc):
# Get the path to bison and m4
BISON_PATH=$(/opt/homebrew/bin/brew --prefix bison)/bin
M4_PATH=$(/opt/homebrew/bin/brew --prefix m4)/bin
# Assemble path
export PATH=$HOME/bin:/opt/homebrew/bin:$BISON_PATH:$M4_PATH:/usr/local/bin:$PATH
These programs are available after the completion of the setup-macos.sh script execution.
Specifically, the MacOS system versions are outdated. These are installed by macOS 14.4 and XCode 15.3 and later. More details are in the Velox README: https://github.com/facebookincubator/velox/blob/3bf1a74042e83981630d24c1eaee190d1f31aac2/README.md#setting-up-on-macos
Example error output:
CMake Error at /opt/homebrew/Cellar/cmake/3.30.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY
OPENSSL_INCLUDE_DIR)
Call Stack (most recent call first):
/opt/homebrew/Cellar/cmake/3.30.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
/opt/homebrew/Cellar/cmake/3.30.2/share/cmake/Modules/FindOpenSSL.cmake:689 (find_package_handle_standard_args)
CMakeLists.txt:176 (find_package)
Explanation A rare issue may occur where the build fails due to OpenSSL not being found even though it was (verifiably) installed with brew.
The issue is that in the path LibreSSL is found. Check the version by running
$ openssl version
If it returns
LibreSSL 3.3.6
then it picks up LibreSSL rather than OpenSSL. LibreSSL may come with the system installation. Some systems appear to have it installed while others do not.
The solution is to make sure that the OpenSSL from homebrew is picked up. Much like with bison and m4 we want that the OpenSSL executable is in the path prior to the system path.
It might be best to generally ensure that /opt/homebrew/bin is added to the path in the zshrc:
export PATH=$HOME/bin:/opt/homebrew/bin:$BISON_PATH:$M4_PATH:/usr/local/bin:$PATH
The solution for ensuring that the correct bison and m4 utilities are found also solves the OpenSSL issue as it added the homebrew bin path already.
Afterwards running OpenSSL should return something similar to
OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024)
Example:
/opt/homebrew/include/fmt/compile.h:153:12: error: call to 'write' is ambiguous
return write<Char>(out, arg);
^~~~~~~~~~~
/opt/homebrew/include/fmt/compile.h:217:15: note: in instantiation of function template specialization 'fmt::detail::field<char, folly::Range<const char *>, 0>::format<std::back_insert_iterator<std::string>, folly::Range<const char *>>' requested here
out = lhs.format(out, args...);
^
This can occur during the dependency build or the Prestissimo/Velox build.
Explanation
The setup-macos.sh script installs a version of fmt from brew. This will be changed in the future.
Recently, the fmt dependency was updated from version 10 to version 11. This is causing the problem. The fmt dependency is installed twice. Once using brew and the other using the source code from github via the install_fmt function in the setup script. Only the version from the setup script is guaranteed to work.
As evident from the error, the headers from the brew version are picked up. Verify the version from brew using
brew info fmt
If it shows something similar to
==> fmt: stable 11.0.2 (bottled), HEAD
the version is not the desired version. It has to be major version 10 (as of this writing - this is subject to change).
Solution
If you are running a new setup from scratch, first, remove fmt from brew
$ brew remove fmt
and then remove the fmt dependency from the list of dependencies to be installed from brew in the setup-macos.sh script. It is in the list named MACOS_VELOX_DEPS.
Then re-run the script. If you encounter the problem because fmt is only installed from brew (maybe you ran the setup-macos.sh script a long time ago) you can selectively install fmt by running
$ setup-macos.sh install_fmt
This works for any dependency that is downloaded and build. You can refer to the script contents for which dependencies can be built and installed this way.
Note, don't forget to set INSTALL_PREFIX and/or DEPENDENCY_DIR if required or used previously for running the script.