Building Julia and LLVM from source is broken on master

[giordano]

Building Julia on 0ef8a91 with the following Make.user

USE_BINARYBUILDER_LLVM = 0
DEPS_GIT = llvm

crashes for me on aarch64-linux-gnu with

[...]
    LINK usr/lib/libjulia-codegen.so.1.12.0
    LINK usr/lib/libjulia-codegen.so
    LINK usr/lib/libjulia-codegen.so.1.12
    JULIA usr/lib/julia/corecompiler.ji

[1217093] signal 6 (-6): Aborted
in expression starting at boot.jl:264
__pthread_kill_implementation at /lib64/libc.so.6 (unknown line)
raise at /lib64/libc.so.6 (unknown line)
abort at /lib64/libc.so.6 (unknown line)
_ZNK4llvm25AMDGPUInstructionSelector15runCustomActionEjRKNS_20GIMatchTableExecutor12MatcherStateE at /data/cceamgi/repo/julia/usr/bin/../lib/libLLVM-17jl.so (unknown line)
registerJITObject at /data/cceamgi/repo/julia/src/debuginfo.cpp:340
unknown function (ip: 0xffffabf83cbf)
unknown function (ip: 0xffffabf83cbf)
Allocations: 0 (Pool: 0; Big: 0); GC: 0
make[1]: *** [sysimage.mk:64: /data/cceamgi/repo/julia/usr/lib/julia/corecompiler.ji] Error 134
make: *** [Makefile:107: julia-sysimg-ji] Error 2

Instead, compilation with the same configuration is successful on d68befd, which is the parent of the merge commit of #55180, which @gbaraldi identified as possible culprit. I don't have the time to do a full git bisect right now, perhaps I'll be able to do it at a later point. Side note, JuliaCI/julia-buildkite#373 also identified #55180 as possible culprit (or at least related).

[giordano]

Confirmed with git bisect

$ cat bisect.sh
#!/bin/bash
make cleanall || true
make -j60 USE_BINARYBUILDER_LLVM=0 DEPS_GIT=llvm
$ git bisect start
$ git bisect good d68befdc0d3479cc3cb8cce64aa8c380253d2f5d
$ git bisect bad 0ef8a91e490dfaa90cb6e4781ca0445c7f991933
$ git bisect start ./bisect.sh

that the first broken commit is fe597c1 is the first bad commit (#55180). CC: @vtjnash.

[vtjnash]

Did you mistakenly build without zlib support? I didn't handle that case, as I didn't think anyone would do that

[giordano]

Did you mistakenly build without zlib support?

I didn't change anything in the build system of LLVM or Julia, I showed above all the options I used.

[giordano]

And also JuliaCI/julia-buildkite#373 shows that a from build source build of julia and llvm has similar issues. It's probably the same issue as this one (all errors are in the same places), but it shows that the standard configuration of the llvm build system within julia is not doing something correctly and this is independently reproducible by multiple users..

[vtjnash]

#47727 ?

[giordano]

What I'm trying to convey is that the default build of llvm from the build system we have here produces a non functional llvm, and now it does so in a dramatic way by breaking Julia build itself. I'm not contesting that a non functional llvm isn't good.

[giordano]

It turned out this was broken by #42622: setting ZLIB_LIBRARY to a directory path is wrong, it should have been the path to the library file itself. A better solution is to use ZLIB_ROOT, which is the CMake-blessed way to suggest where Zlib is installed.

[vtjnash]

I don't have a build for testing this, but this should also work to disable zlib compression when not available:

diff --git a/src/debuginfo.cpp b/src/debuginfo.cpp
index 8455081107..53081bd77a 100644
--- a/src/debuginfo.cpp
+++ b/src/debuginfo.cpp
@@ -337,9 +337,16 @@ void JITDebugInfoRegistry::registerJITObject(const object::ObjectFile &Object,
 #endif // defined(_OS_WINDOWS_)
 
     SmallVector<uint8_t, 0> packed;
-    compression::zlib::compress(ArrayRef<uint8_t>((uint8_t*)Object.getData().data(), Object.getData().size()), packed, compression::zlib::DefaultCompression);
+    auto uncompressedsize = Object.getData().size();
+    if (compression::zlib::isAvailable())
+        compression::zlib::compress(ArrayRef<uint8_t>((uint8_t*)Object.getData().data(), uncompressedsize), packed, compression::zlib::DefaultCompression);
+    else {
+        packed.append(Object.getData().begin(), Object.getData().end());
+        uncompressedsize = 0;
+    }
     jl_jit_add_bytes(packed.size());
-    auto ObjectCopy = new LazyObjectInfo{packed, Object.getData().size()}; // intentionally leaked so that we don't need to ref-count it, intentionally copied so that we exact-size the allocation (since no shrink_to_fit function)
+    auto ObjectCopy = new LazyObjectInfo{packed, uncompressedsize}; // intentionally leaked so that we don't need to ref-count it, intentionally copied so that we exact-size the allocation (since no shrink_to_fit function)
+
     auto symbols = object::computeSymbolSizes(Object);
     bool hassection = false;
     for (const auto &sym_size : symbols) {

[giordano]

I don't have a build for testing this

That's easy to resolve: just run make -j USE_BINARYBUILDER_LLVM=0 🙂

but this should also work to disable zlib compression when not available:

I can confirm this doesn't make julia compilation fail with USE_BINARYBUILDER_LLVM=0. I think #55354 is still worthwhile to get in because we're passing wrong options around, but checking if zlib is actually available sounds good to do too.