-
Notifications
You must be signed in to change notification settings - Fork 951
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fails to build with Xcode 10 on macOS Mojave, openssl and yaracpp issue #439
Comments
Hi. As for the As for the |
UPD: reinstalling openssl and setting some env vars as suggested by brew did work. Some yara issues were fixed by reinstalling flex and bison (and setting env vars), setting Apple specific env vars as suggested by README, and reinstalling command-line tools. Current status:
|
I definitely had the CLI tools installed, and also tried reinstalling them just to be sure. Setting up Homebrew OpenSSL does get around the OpenSSL build failure: export PATH="/usr/local/opt/openssl/bin:$PATH"
export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig" But it still fails on yaracpp:
|
One thing I should point out, the issue is actually with Xcode 10 (which dropped 32-bit support) rather than Mojave itself. If you were to install the last version of Xcode 9 and the accompanying CLI tools on Mojave you can build it. I was successfully able to build it in a Mojave VM with Xcode 9 and copy over the builds to my host OS. |
If I remember correctly, this may have something to do with a homebrew installed version of binutils. I think I had a problem with it using binutils version during building, but macos tool for linking. Or maybe vise-versa. Anyway, uninstalling via "brew remove binutils" fixed it for me. Apparently something changed somewhere.Try it and let us know. I remember seeing Command failed error of 'make' '-jN'. Hopefully this saves you from downgrading xcode. |
@AlexanderOMara Can you please attach the output from |
Attached: |
Thank you. Could you please try the following builds of YARA (our own fork and the upstream version) and let us know whether each of the builds succeeded/failed? It will help us to determine whether the issue is with our own fork or with the upstream version, and if so, whether it has been fixed.
|
Pass:
Fail:
|
Are you sure that the |
I thought that was strange too, but I just tested it again, and it passed. |
@bmourit I just tried |
Alright. Could I ask you to do one final test?
That |
I guess something fishy is going on, because that passed too. |
@metthal Do you have an idea what might be going on here? |
@AlexanderOMara Did you clean all build cache first? I had to delete the build folder and make a new one and build everything fresh. |
@bmourit Yep, I always deleted the build folder between build attempts. |
Hmm.. odd. I had an issue with those exact same external projects (yaracpp and openssl). I'm trying to remember exactly what I did to fix it. There was something to do with my $PATH as well. I'll see if I can dig around some more to refresh my memory. |
@AlexanderOMara |
|
Try this: Update: |
This happens to me too |
@AlexanderOMara @dendisuhubdy Could you please try performing the following two builds and let us know if they pass or fail? The results will help us to determine whether the issue is in the currently used version of YARA in RetDec/yaracpp or in something else. Note: Before you try the builds, please ensure that you have command-line tools installed and the following variables set:
If any of the builds fail, please attach |
@s3rvac both failed, here are the build log dump on Pastebin (unlisted) https://pastebin.com/bX9r9cmf and https://pastebin.com/G6j5fQsf |
@dendisuhubdy Thank you. Could you please try installing SDK headers by running
and then re-trying the first yaracpp build above from scratch (from a clean directory)? |
EDIT: Please run this even if the workaround above worked for you. We would like to get rid of the requirement of installing command line tools and this would help us understand why is it required. If the step above didn't work for you, I would like to ask you (or anyone else in this thread who is having this problem) to run this to build
And then run this to build upstream yara:
The first build should fail and second one should pass. Now, please attach these two files:
|
@s3rvac it says that the grokmachine@Dendis-MacBook-Pro ~ open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg ✔ 4681 18:05:46
The file /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg does not exist. |
@dendisuhubdy Go to https://developer.apple.com/download/more/ and download the "Command Line Tools (macOS 10.14) for Xcode 10.1" pkg. |
@bmourit I did that, downloaded it and installed it but honestly I do remember installing the command line tool when I installed Xcode 10.1 already. And I redid what @s3rvac told me to and it still has linking issues. Take a look libtool: link: ar cru .libs/libyara.a modules/tests.o modules/pe.o modules/elf.o modules/macho.o modules/math.o modules/pe_utils.o grammar.o ahocorasick.o arena.o atoms.o compiler.o endian.o exec.o exefiles.o filemap.o hash.o hex_grammar.o hex_lexer.o lexer.o libyara.o mem.o modules.o object.o parser.o proc.o re.o re_grammar.o re_lexer.o rules.o scan.o sizedstr.o strutils.o stream.o threading.o
ar: `u' modifier ignored since `D' is the default (see `U')
libtool: link: ranlib .libs/libyara.a
libtool: link: ( cd ".libs" && rm -f "libyara.la" && ln -s "../libyara.la" "libyara.la" )
libtool: link: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -std=gnu99 -Wall -I./libyara/include -O3 -I/opt/local/include -I/usr/local/opt/openssl/include -D_THREAD_SAFE -o yarac args.o yarac.o -L/usr/local/opt/llvm/lib libyara/.libs/libyara.a -lm
ld: warning: ignoring file libyara/.libs/libyara.a, file was built for archive which is not the architecture being linked (x86_64): libyara/.libs/libyara.a
Undefined symbols for architecture x86_64:
"_yr_compiler_add_file", referenced from:
_main in yarac.o
"_yr_compiler_create", referenced from:
_main in yarac.o
"_yr_compiler_define_boolean_variable", referenced from:
_define_external_variables in yarac.o
"_yr_compiler_define_integer_variable", referenced from:
_define_external_variables in yarac.o
"_yr_compiler_define_string_variable", referenced from:
_define_external_variables in yarac.o
"_yr_compiler_destroy", referenced from:
_main in yarac.o
"_yr_compiler_get_rules", referenced from:
_main in yarac.o
"_yr_compiler_set_callback", referenced from:
_main in yarac.o
"_yr_finalize", referenced from:
_main in yarac.o
"_yr_initialize", referenced from:
_main in yarac.o
"_yr_rules_destroy", referenced from:
_main in yarac.o
"_yr_rules_save", referenced from:
_main in yarac.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[4]: *** [yarac] Error 1
make[4]: *** Waiting for unfinished jobs....
libtool: link: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -std=gnu99 -Wall -I./libyara/include -O3 -I/opt/local/include -I/usr/local/opt/openssl/include -D_THREAD_SAFE -o yara args.o threading.o yara.o -L/usr/local/opt/llvm/lib libyara/.libs/libyara.a -lm
ld: warning: ignoring file libyara/.libs/libyara.a, file was built for archive which is not the architecture being linked (x86_64): libyara/.libs/libyara.a
Undefined symbols for architecture x86_64:
"_yr_compiler_add_file", referenced from:
_main in yara.o
"_yr_compiler_create", referenced from:
_main in yara.o
"_yr_compiler_define_boolean_variable", referenced from:
_define_external_variables in yara.o
"_yr_compiler_define_float_variable", referenced from:
_define_external_variables in yara.o
"_yr_compiler_define_integer_variable", referenced from:
_define_external_variables in yara.o
"_yr_compiler_define_string_variable", referenced from:
_define_external_variables in yara.o
"_yr_compiler_destroy", referenced from:
_main in yara.o
"_yr_compiler_get_rules", referenced from:
_main in yara.o
"_yr_compiler_set_callback", referenced from:
_main in yara.o
"_yr_filemap_map", referenced from:
_load_modules_data in yara.o
"_yr_filemap_unmap", referenced from:
_unload_modules_data in yara.o
_main in yara.o
"_yr_finalize", referenced from:
_main in yara.o
"_yr_finalize_thread", referenced from:
_scanning_thread in yara.o
"_yr_get_tidx", referenced from:
_handle_message in yara.o
"_yr_initialize", referenced from:
_main in yara.o
"_yr_object_print_data", referenced from:
_callback in yara.o
"_yr_rules_define_boolean_variable", referenced from:
_define_external_variables in yara.o
"_yr_rules_define_float_variable", referenced from:
_define_external_variables in yara.o
"_yr_rules_define_integer_variable", referenced from:
_define_external_variables in yara.o
"_yr_rules_define_string_variable", referenced from:
_define_external_variables in yara.o
"_yr_rules_destroy", referenced from:
_main in yara.o
"_yr_rules_load", referenced from:
_main in yara.o
"_yr_rules_scan_file", referenced from:
_scanning_thread in yara.o
_main in yara.o
"_yr_rules_scan_proc", referenced from:
_main in yara.o
"_yr_set_configuration", referenced from:
_main in yara.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[4]: *** [yara] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [deps/yara/src/yara-stamp/yara-build] Error 2
make[1]: *** [deps/CMakeFiles/yara.dir/all] Error 2
make: *** [all] Error 2 |
@dendisuhubdy what's your output of the following: |
I am having the same issues.. watching this thread closely; let me know if I can help. |
You can try what I mentioned in this comment. It would help us with finding differences between direct build of |
grokmachine@Dendis-MacBook-Pro ~/dev/bitwyre/exchange/bitcoin_dev/bips master which -a ar ✔ 5292 00:12:00
/usr/local/bin/ar
/usr/bin/ar
grokmachine@Dendis-MacBook-Pro ~/dev/bitwyre/exchange/bitcoin_dev/bips master which -a ranlib ✔ 5293 00:12:53
/usr/local/bin/ranlib
/usr/bin/ranlib |
I don't even get far enough to have the build logs you're looking for. I get a libtoolize error in both cases: yaracpp:
yara:
For some additional environment info, I'm running 10.14.2:
|
I added a symlink to glibtoolize to hack around that error... Two build logs attached as requested. |
Ok, success... The workaround above to install the SDK headers seems to have worked. Note, these are apparently not installed as part of the command line tools normally, which I had installed already.
|
@dendisuhubdy |
Thank you @appleguru for the files. I have prepared branch
|
@bmourit hmm weird ok thanks! Lemme add the correct Xcode versions of ranlib and ar |
For everyone still having this issue: Could you please try the steps from this comment to see whether it resolves the issue? If not, could you please give us the full output and log files? |
Hi, I had the same issue and I've been able to fix it by these steps: OpenSSL issue occurred because export OPENSSL_ROOT_DIR=/usr/local//Cellar/openssl/1.0.2s/
export OPENSSL_LIBRARIES=/usr/local//Cellar/openssl/1.0.2s/lib/ Then I stumbled on the same issue with To fix this I reinstalled macOS SDK headers: open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg After this, I was able to install RetDec successfully. Edit:
|
This should fix some (not all) macOS compilation issues from #439.
* deps/openssl: bump version from 1.1.0f to 1.1.1c. This should fix some (not all) macOS compilation issues from #439. * deps/openssl: add Threads to openssl-crypto's target_link_libraries().
OpenSSL-build-related issues should be fixed by #601 (already merged into |
The yaracpp-related issues should be fixed by #611 (already merged into |
I've been trying to get this to build with Xcode 10 on macOS Mojave, but it seems to be running into trouble with 2 of the dependencies.
...
Attached are the log files detailing the errors.
openssl-build-err.log
yaracpp-project-build-err.log
Based on the log file info, I suspect this issue may be related to Apple dropping support for 32-bit code.
The text was updated successfully, but these errors were encountered: