ASan link order error with Bazel 6.1.0 and non-Xcode toolchain

[fmeum]

Description of the bug:

Since 0f9912b, which is included in Bazel 6.1.0 but not 6.0.0, ASan builds that don't use the Xcode toolchain fail at runtime with:

==38681==ERROR: Interceptors are not working. This may be because AddressSanitizer is loaded too late (e.g. via dlopen). Please launch the executable with:
DYLD_INSERT_LIBRARIES=/opt/homebrew/Cellar/llvm/15.0.2/lib/clang/15.0.2/lib/darwin/libclang_rt.asan_osx_dynamic.dylib

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

git clone https://github.com/CodeIntelligenceTesting/cifuzz
cd cifuzz
git checkout 3dad1be36cfbd05a4b245e46cf8fe42cb6f3730e
cd examples/bazel
USE_BAZEL_VERSION=0f9912ba8a6780a124478fd3ff4fb91b64dbae53 bazelisk run --repo_env=CC=clang --repo_env=CXX=clang++ --repo_env=BAZEL_USE_CPP_ONLY_TOOLCHAIN=1 --copt -g --compilation_mode=opt --copt -Og --copt -UNDEBUG --copt -U_FORTIFY_SOURCE --@rules_fuzzing//fuzzing:cc_engine=@rules_fuzzing//fuzzing/engines:libfuzzer --@rules_fuzzing//fuzzing:cc_engine_instrumentation=libfuzzer --@rules_fuzzing//fuzzing:cc_engine_sanitizer=asan --@rules_fuzzing//fuzzing:cc_engine_sanitizer=ubsan --@cifuzz//:__internal_has_libfuzzer --verbose_failures //src:explore_me_fuzz_test_bin

Which operating system are you running Bazel on?

macOS

What is the output of bazel info release?

6.1.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[fmeum]

@keith @kshyanashree This is breaking Bazel support in our open-source tool with 6.1.0. It would be great if we could get a bugfix release.

[fmeum]

@bazel-io flag

[meteorcloudy]

Looks like 0f9912b is a regression, we probably should do a patch release. /cc @keith @brentleyjones @buildbreaker2021

[keith]

@meteorcloudy lets just revert out of 6.x (personally I didn't care if it was in there at all), and then try to do some fix at HEAD

[meteorcloudy]

Sounds fine, but 6.1.1 can definitely go out much faster than 6.2, so let's revert that change in a patch release (as if we never cherry picked it).

[keith]

Also note I'm trusting the report here since I don't know off hand why this would cause this issue

[fmeum]

I also don't know why this flag would break us and definitely don't understand it well enough to even risk a guess. Our example is very simple and works as expected on Linux (clang) and Windows (MSVC and clang-cl) as well as macOS with Bazel 6, so I leaned towards "blaming" the change that added the toolchain flag.

@keith Happy to look into the root cause of this together at some point.

[meteorcloudy]

@fmeum Can you confirm reverting that change in 6.1 fixes the regression?

[fmeum]

@meteorcloudy Yes, I can confirm that it does.

[keith]

after looking into this a bit with fabian i think it's on the consumer side, but we can revert for 6.x still and let folks fix in 7.x instead

[ShreeM01]

@bazel-io fork 6.1.1

[keertk]

Reverted in 6.1.1 - #17696

[keertk]

@bazel-io fork 6.2.0