AFL seems to be failing to compile

[evverx]

Looks like all the projects where AFL is used are failing to compile with something like

Step #3 - "compile-afl-address-x86_64": meson.build:3:0: ERROR: Unable to get clang pre-processor defines:
Step #3 - "compile-afl-address-x86_64": error: unable to load plugin '/src/aflplusplus/afl-llvm-pass.so': '/src/aflplusplus/afl-llvm-pass.so: cannot open shared object file: No such file or directory'
Step #3 - "compile-afl-address-x86_64": 
Step #3 - "compile-afl-address-x86_64": + exit 1
Step #3 - "compile-afl-address-x86_64": ********************************************************************************
Step #3 - "compile-afl-address-x86_64": Failed to build.
Step #3 - "compile-afl-address-x86_64": To reproduce, run:
Step #3 - "compile-afl-address-x86_64": python infra/helper.py build_image systemd
Step #3 - "compile-afl-address-x86_64": python infra/helper.py build_fuzzers --sanitizer address --engine afl --architecture x86_64 systemd

Step #3 - "compile-afl-address-x86_64": + ZLIB_DIR=/usr/lib/x86_64-linux-gnu
Step #3 - "compile-afl-address-x86_64": + /src/aflplusplus/afl-clang-fast -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -D_GNU_SOURCE -DHAVE_CONFIG_H -I. -I./lib -I./libelf -I./libebl -I./libdw -I./libdwelf -I./libdwfl -I./libasm -c /src/fuzz-dwfl-core.c -o fuzz-dwfl-core.o
Step #3 - "compile-afl-address-x86_64": error: unable to load plugin '/src/aflplusplus/afl-llvm-dict2file.so': '/src/aflplusplus/afl-llvm-dict2file.so: cannot open shared object file: No such file or directory'
Step #3 - "compile-afl-address-x86_64": error: unable to load plugin '/src/aflplusplus/cmplog-switches-pass.so': '/src/aflplusplus/cmplog-switches-pass.so: cannot open shared object file: No such file or directory'
Step #3 - "compile-afl-address-x86_64": error: unable to load plugin '/src/aflplusplus/split-switches-pass.so': '/src/aflplusplus/split-switches-pass.so: cannot open shared object file: No such file or directory'
Step #3 - "compile-afl-address-x86_64": error: unable to load plugin '/src/aflplusplus/SanitizerCoveragePCGUARD.so': '/src/aflplusplus/SanitizerCoveragePCGUARD.so: cannot open shared object file: No such file or directory'
Step #3 - "compile-afl-address-x86_64": error: unable to load plugin '/src/aflplusplus/cmplog-instructions-pass.so': '/src/aflplusplus/cmplog-instructions-pass.so: cannot open shared object file: No such file or directory'
Step #3 - "compile-afl-address-x86_64": error: unable to load plugin '/src/aflplusplus/cmplog-routines-pass.so': '/src/aflplusplus/cmplog-routines-pass.so: cannot open shared object file: No such file or directory'
Step #3 - "compile-afl-address-x86_64": ********************************************************************************
Step #3 - "compile-afl-address-x86_64": Failed to build.
Step #3 - "compile-afl-address-x86_64": To reproduce, run:
Step #3 - "compile-afl-address-x86_64": python infra/helper.py build_image elfutils
Step #3 - "compile-afl-address-x86_64": python infra/helper.py build_fuzzers --sanitizer address --engine afl --architecture x86_64 elfutils
Step #3 - "compile-afl-address-x86_64": ********************************************************************************
Finished Step #3 - "compile-afl-address-x86_64"
ERROR
ERROR: build step 3 "gcr.io/oss-fuzz/elfutils" failed: step exited with non-zero status: 1

Step #3 - "compile-afl-address-x86_64": + ./configure --disable-all-programs --enable-libuuid --enable-libfdisk --enable-last --enable-fuzzing-engine --enable-libmount --enable-libblkid
Step #3 - "compile-afl-address-x86_64": checking for gcc... /src/aflplusplus/afl-clang-fast
Step #3 - "compile-afl-address-x86_64": checking whether the C compiler works... no
Step #3 - "compile-afl-address-x86_64": configure: error: in `/src/util-linux':
Step #3 - "compile-afl-address-x86_64": configure: error: C compiler cannot create executables
Step #3 - "compile-afl-address-x86_64": See `config.log' for more details
Step #3 - "compile-afl-address-x86_64": ********************************************************************************
Step #3 - "compile-afl-address-x86_64": Failed to build.
Step #3 - "compile-afl-address-x86_64": To reproduce, run:
Step #3 - "compile-afl-address-x86_64": python infra/helper.py build_image util-linux
Step #3 - "compile-afl-address-x86_64": python infra/helper.py build_fuzzers --sanitizer address --engine afl --architecture x86_64 util-linux
Step #3 - "compile-afl-address-x86_64": ********************************************************************************
Finished Step #3 - "compile-afl-address-x86_64"
ERROR
ERROR: build step 3 "gcr.io/oss-fuzz/util-linux" failed: step exited with non-zero status: 1

[DavidKorczynski]

CC @vanhauser-thc

[vanhauser-thc]

well the current LLVM state seems to have made a huge change - removing/renaming IRB.CreateLoad().
Not sure what they are smoking but this is a huge fuckup on their side IMHO.
dunno if this is a bug or if that change is really thought through and will not be rolled backwards.

to be honest - this is not something we will change the next weeks, because the impact is huge and we first have to see if this change stays or not.

so intermediate solution: pin to the previous llvm commit and get back to this in 2-4 weeks.

@jonathanmetzman

[jonathanmetzman]

well the current LLVM state seems to have made a huge change - removing/renaming IRB.CreateLoad(). Not sure what they are smoking but this is a huge fuckup on their side IMHO. dunno if this is a bug or if that change is really thought through and will not be rolled backwards.

to be honest - this is not something we will change the next weeks, because the impact is huge and we first have to see if this change stays or not.

so intermediate solution: pin to the previous llvm commit and get back to this in 2-4 weeks.

@jonathanmetzman

ACK. Going to do this ASAP

[nikic]

well the current LLVM state seems to have made a huge change - removing/renaming IRB.CreateLoad(). Not sure what they are smoking but this is a huge fuckup on their side IMHO. dunno if this is a bug or if that change is really thought through and will not be rolled backwards.

to be honest - this is not something we will change the next weeks, because the impact is huge and we first have to see if this change stays or not.

so intermediate solution: pin to the previous llvm commit and get back to this in 2-4 weeks.

Please see https://llvm.org/docs/OpaquePointers.html for broader context on the changes. The tl;dr as far as IRBuilder is concerned is that you explicitly need to pass the loaded type / the GEP base type to IRBuilder methods.

As this seems to have caught you unawares, you are probably ignoring deprecation warnings in your build -- I recommend building with -Werror=deprecated-declarations if your build is not otherwise warning-clean to avoid missing them.

[jonathanmetzman]

well the current LLVM state seems to have made a huge change - removing/renaming IRB.CreateLoad(). Not sure what they are smoking but this is a huge fuckup on their side IMHO. dunno if this is a bug or if that change is really thought through and will not be rolled backwards.
to be honest - this is not something we will change the next weeks, because the impact is huge and we first have to see if this change stays or not.
so intermediate solution: pin to the previous llvm commit and get back to this in 2-4 weeks.

Please see https://llvm.org/docs/OpaquePointers.html for broader context on the changes. The tl;dr as far as IRBuilder is concerned is that you explicitly need to pass the loaded type / the GEP base type to IRBuilder methods.

As this seems to have caught you unawares, you are probably ignoring deprecation warnings in your build -- I recommend building with -Werror=deprecated-declarations if your build is not otherwise warning-clean to avoid missing them.

Thanks for this tip

[vanhauser-thc]

I updated all passes from afl++ to work with the most current llvm 14 commit.
currentl running through fuzzbench to see if there are issues (but this is not a good test because fuzzbench is sadly on llvm 12, and fix in afl++ is if guarded for LLVM >= 14). I do not have llvm 12 installed or available on any system I use.

As this seems to have caught you unawares, you are probably ignoring deprecation warnings in your build -- I recommend building with -Werror=deprecated-declarations if your build is not otherwise warning-clean to avoid missing them.

we support llvm down to 3.3 ... it is a possible option but would require so much work :(