Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resurrect libunwind patches #45189

Merged
merged 1 commit into from
May 17, 2022
Merged

Resurrect libunwind patches #45189

merged 1 commit into from
May 17, 2022

Conversation

fxcoudert
Copy link
Contributor

Testing in progress…
I hope this can fix #44499

@fxcoudert
Copy link
Contributor Author

@giordano this allows the patches to apply to 1.8.0-julia3, but then immediately afterwards, a new error:

cd scratch/llvmunwind-12.0.1/ && \
	cmake /private/tmp/julia-20220504-59800-1afzb6i/julia-1.8.0-beta3/deps/srccache/llvmunwind-12.0.1/ -DCMAKE_INSTALL_PREFIX:PATH=/private/tmp/julia-20220504-59800-1afzb6i/julia-1.8.0-beta3/usr -DCMAKE_PREFIX_PATH=/private/tmp/julia-20220504-59800-1afzb6i/julia-1.8.0-beta3/usr -DCMAKE_INSTALL_LIBDIR=/private/tmp/julia-20220504-59800-1afzb6i/julia-1.8.0-beta3/usr/lib -DCMAKE_INSTALL_BINDIR=/private/tmp/julia-20220504-59800-1afzb6i/julia-1.8.0-beta3/usr/bin -DLIB_INSTALL_DIR=/private/tmp/julia-20220504-59800-1afzb6i/julia-1.8.0-beta3/usr/lib -DCMAKE_MACOSX_RPATH=1 -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_C_COMPILER="$(which clang)" -DCMAKE_C_COMPILER_ARG1="-mmacosx-version-min=12" -DCMAKE_CXX_COMPILER="clang++" -DCMAKE_CXX_COMPILER_ARG1="-mmacosx-version-min=12" -DCMAKE_LINKER="$(which ld)" -DCMAKE_AR="$(which ar)" -DCMAKE_RANLIB="$(which ranlib)" -DCMAKE_BUILD_TYPE=MinSizeRel -DLIBUNWIND_ENABLE_PEDANTIC=OFF -DLLVM_CONFIG_PATH=/private/tmp/julia-20220504-59800-1afzb6i/julia-1.8.0-beta3/usr/tools/llvm-config
Re-run cmake no build system arguments
CMake Error at CMakeLists.txt:2 (message):
  libunwind requires being built in a monorepo layout with libcxx available


-- Configuring incomplete, errors occurred!
make[2]: *** [scratch/llvmunwind-12.0.1/build-configured] Error 1

This is never-ending…

@giordano
Copy link
Contributor

giordano commented May 4, 2022

In Yggdrasil we have https://github.com/JuliaPackaging/Yggdrasil/blob/5b9b2eaa754ebf244e9e0bcdda1e8e60f55b9108/L/LLVMLibUnwind/LLVMLibUnwind%4012.0.1/bundled/patches/0003-revert-monorepo-requirement.patch, which sounds relevant.

@giordano giordano added building Build system, or building Julia or its dependencies system:mac Affects only macOS external dependencies Involves LLVM, OpenBLAS, or other linked libraries backport 1.8 Change should be backported to release-1.8 labels May 4, 2022
@fxcoudert
Copy link
Contributor Author

But why are patches inside the source repo not the same as the patches inside the builder software (if I understand correctly what Yggdrasil is)?

@fxcoudert fxcoudert force-pushed the patches branch 2 times, most recently from 7d96257 to 58dac77 Compare May 4, 2022 17:25
@fxcoudert
Copy link
Contributor Author

fxcoudert commented May 4, 2022 •
edited
Loading

Two more patches, yet another build failure:

mkdir -p scratch/llvmunwind-12.0.1/
cd scratch/llvmunwind-12.0.1/ && \
	cmake /private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/deps/srccache/llvmunwind-12.0.1/ -DCMAKE_INSTALL_PREFIX:PATH=/private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr -DCMAKE_PREFIX_PATH=/private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr -DCMAKE_INSTALL_LIBDIR=/private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr/lib -DCMAKE_INSTALL_BINDIR=/private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr/bin -DLIB_INSTALL_DIR=/private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr/lib -DCMAKE_MACOSX_RPATH=1 -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_C_COMPILER="$(which clang)" -DCMAKE_C_COMPILER_ARG1="-mmacosx-version-min=12" -DCMAKE_CXX_COMPILER="clang++" -DCMAKE_CXX_COMPILER_ARG1="-mmacosx-version-min=12" -DCMAKE_LINKER="$(which ld)" -DCMAKE_AR="$(which ar)" -DCMAKE_RANLIB="$(which ranlib)" -DCMAKE_BUILD_TYPE=MinSizeRel -DLIBUNWIND_ENABLE_PEDANTIC=OFF -DLLVM_CONFIG_PATH=/private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr/tools/llvm-config
Re-run cmake no build system arguments
-- The C compiler identification is AppleClang 13.1.6.13160021
 49  237k   49  118k    0     0  53259      0  0:00:04  0:00:02  0:00:02  154k-- The CXX compiler identification is AppleClang 13.1.6.13160021
  0 4867k    0 18821    0     0   7935      0  0:10:28  0:00:02  0:10:26 22896-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: /opt/homebrew/Library/Homebrew/shims/mac/super/clang
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/homebrew/Library/Homebrew/shims/mac/super/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/homebrew/Library/Homebrew/shims/mac/super/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found LLVM_CONFIG_PATH as /private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr/tools/llvm-config
-- /private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr/tools/llvm-config --includedir --prefix --src-root
CMake Error at CMakeLists.txt:39 (message):
  llvm-config failed with status No such file or directory

There is no llvm-config in /private/tmp/julia-20220504-68791-empngk/julia-1.8.0-beta3/usr/tools/, so I do not know why it said Found LLVM_CONFIG_PATH as […]

@fxcoudert
Copy link
Contributor Author

The answer is above: the makefile passes an explicit path to LLVM_CONFIG_PATH, but that path is nonexistent 🤷

@fxcoudert
Copy link
Contributor Author

Shouldn't there be a dependency so that llvm is built before llvmunwind? I don't find it, and I think it would make sense.

In deps/unwind.mk I can add to the target $(BUILDDIR)/llvmunwind-$(LLVMUNWIND_VER)/build-configured a dependency on $(LLVM_BUILDDIR_withtype)/build-compiled. It seems to fix the build in my specific case, but I'm not sure how robust that is and whether that's the right solution.

@giordano any advice? I'm really trying to help, but the build system is so complex and so fragile, it's hard to know.

@fxcoudert
Copy link
Contributor Author

So, with:

I can actually compile Julia 1.8.0-rc3 from source, on aarch64-apple-darwin21, with its own LLVM/libunwind. It passes the previous sore point (Core.Compiler). It does get stuck a bit later:

Sysimage built. Summary:
Total ───────  39.094312 seconds 
Base: ───────  15.715046 seconds 40.1978%
Stdlibs: ────  23.378290 seconds 59.7997%
Warning: git information unavailable; versioning information limited
 cd /private/tmp/julia-20220504-75031-1vwosmq/julia-1.8.0-beta3/base && if ! JULIA_BINDIR=/private/tmp/julia-20220504-75031-1vwosmq/julia-1.8.0-beta3/usr/bin WINEPATH="/private/tmp/julia-20220504-75031-1vwosmq/julia-1.8.0-beta3/usr/bin;$WINEPATH" JULIA_NUM_THREADS=1  /private/tmp/julia-20220504-75031-1vwosmq/julia-1.8.0-beta3/usr/bin/julia -O3 -C "generic" --output-o /private/tmp/julia-20220504-75031-1vwosmq/julia-1.8.0-beta3/usr/lib/julia/sys-o.a.tmp  --startup-file=no --warn-overwrite=yes --sysimage /private/tmp/julia-20220504-75031-1vwosmq/julia-1.8.0-beta3/usr/lib/julia/sys.ji /private/tmp/julia-20220504-75031-1vwosmq/julia-1.8.0-beta3/contrib/generate_precompile.jl 1; then echo '*** This error is usually fixed by running `make clean`. If the error persists, try `make cleanall`. ***'; false; fi 
Generating REPL precompile statements... 25/40

It's been stuck there for a long while. It was using CPU at the beginning of this process, reached 25, then crashed:

par défaut	22:42:25.841439+0200	kernel	Data/Stack execution not permitted: julia[pid 23507] at virtual address 0x1026ac000, protections were read-write
par défaut	22:42:25.841595+0200	kernel	Data/Stack execution not permitted: julia[pid 23507] at virtual address 0x1026ac000, protections were read-write
par défaut	22:42:25.842495+0200	mDNSResponder	[R124304] DNSServiceCreateConnection STOP PID[23507](julia)
par défaut	22:42:25.841638+0200	kernel	AMFI: Denying core dump for pid 23507 (julia)
par défaut	22:42:25.841650+0200	kernel	julia[23507] Corpse allowed 1 of 5

The crash report is the following:

-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Process:               julia [23507]
Path:                  /private/tmp/*/julia
Identifier:            julia
Version:               ???
Code Type:             ARM-64 (Native)
Parent Process:        julia [23500]
Responsible:           Terminal [573]
User ID:               502

Date/Time:             2022-05-04 22:42:25.8507 +0200
OS Version:            macOS 12.3.1 (21E258)
Report Version:        12
Anonymous UUID:        2D7F24AB-FBBA-6FE0-0F16-A8B281CB69B5

Sleep/Wake UUID:       9CEDEA75-FD5B-4B27-BCCD-57FB53ABAA3D

Time Awake Since Boot: 150000 seconds
Time Since Wake:       2328 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGBUS)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x00000001026ac008
Exception Codes:       0x0000000000000002, 0x00000001026ac008
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace SIGNAL, Code 10 Bus error: 10
Terminating Process:   exc handler [23507]

VM Region Info: 0x1026ac008 is in 0x1026ac000-0x1028ac000;  bytes after start: 8  bytes before end: 2097143
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      __LINKEDIT                  1026a4000-1026ac000    [   32K] r--/rwx SM=COW  ...8proc.2.dylib
--->  VM_ALLOCATE                 1026ac000-1028ac000    [ 2048K] rw-/rwx SM=PRV  
      VM_ALLOCATE                 1028ac000-1028b0000    [   16K] r-x/rwx SM=COW  

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   ???                           	       0x1026ac008 ???
1   ???                           	       0x29b06c3d8 ???
2   ???                           	       0x29b109420 ???
3   ???                           	       0x294832dc8 ???
4   ???                           	       0x298d60d34 ???
5   ???                           	       0x294b94354 ???
6   ???                           	       0x2901d00bc ???
7   libjulia-internal.1.8.dylib   	       0x10290d18c do_call + 188
8   libjulia-internal.1.8.dylib   	       0x10290bb48 eval_body + 1464
9   libjulia-internal.1.8.dylib   	       0x10290c17c jl_interpret_toplevel_thunk + 264
10  libjulia-internal.1.8.dylib   	       0x102923cfc jl_toplevel_eval_flex + 4952
11  libjulia-internal.1.8.dylib   	       0x102923bd8 jl_toplevel_eval_flex + 4660
12  libjulia-internal.1.8.dylib   	       0x102924920 ijl_toplevel_eval_in + 156
13  ???                           	       0x11eb104a4 ???
14  ???                           	       0x119e0092c ???
15  libjulia-internal.1.8.dylib   	       0x102901cf8 jl_f__call_latest + 76
16  ???                           	       0x119c98e80 ???
17  ???                           	       0x119cb3270 ???
18  ???                           	       0x119cc05b0 ???
19  libjulia-internal.1.8.dylib   	       0x102942d8c true_main + 188
20  libjulia-internal.1.8.dylib   	       0x102942c84 jl_repl_entrypoint + 180
21  julia                         	       0x10233bf9c main + 12
22  dyld                          	       0x10244d088 start + 516

Thread 1:
0   libsystem_kernel.dylib        	       0x1ae10ec40 kevent + 8
1   libjulia-internal.1.8.dylib   	       0x1029442f0 signal_listener + 196
2   libsystem_pthread.dylib       	       0x1ae14626c _pthread_start + 148
3   libsystem_pthread.dylib       	       0x1ae14108c thread_start + 8

Thread 2:
0   libsystem_kernel.dylib        	       0x1ae1088d0 mach_msg_trap + 8
1   libsystem_kernel.dylib        	       0x1ae108d40 mach_msg + 76
2   libsystem_kernel.dylib        	       0x1ae110638 mach_msg_server + 320
3   libjulia-internal.1.8.dylib   	       0x1029432e4 mach_segv_listener + 40
4   libsystem_pthread.dylib       	       0x1ae14626c _pthread_start + 148
5   libsystem_pthread.dylib       	       0x1ae14108c thread_start + 8

Thread 3:
0   libsystem_kernel.dylib        	       0x1ae1088d0 mach_msg_trap + 8
1   libsystem_kernel.dylib        	       0x1ae108d40 mach_msg + 76
2   CoreFoundation                	       0x1ae212c1c __CFRunLoopServiceMachPort + 372
3   CoreFoundation                	       0x1ae2110cc __CFRunLoopRun + 1180
4   CoreFoundation                	       0x1ae210490 CFRunLoopRunSpecific + 600
5   CoreFoundation                	       0x1ae29d628 CFRunLoopRun + 64
6   DebugSymbols                  	       0x1c0f0735c SpotlightQueryThread(void*) + 564
7   libsystem_pthread.dylib       	       0x1ae14626c _pthread_start + 148
8   libsystem_pthread.dylib       	       0x1ae14108c thread_start + 8

Thread 4:
0   libsystem_pthread.dylib       	       0x1ae141078 start_wqthread + 0

Thread 5:
0   libsystem_pthread.dylib       	       0x1ae141078 start_wqthread + 0


Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x000000016dabd888   x1: 0x000000010b028070   x2: 0x0000000000000000   x3: 0x0000000000000000
    x4: 0xffffa001077ffad0   x5: 0x0000000000000020   x6: 0x0000000000000800   x7: 0x0000000000000000
    x8: 0x00000001026ac008   x9: 0x000000016952df48  x10: 0x00000000000007fb  x11: 0x00000000000007fd
   x12: 0x0000000000000013  x13: 0x00000000f44608b3  x14: 0x00000000f4661000  x15: 0x00000000000000c2
   x16: 0x0000000000000021  x17: 0x0000000074600000  x18: 0x0000000000000000  x19: 0x000000016effad10
   x20: 0x000000010b028070  x21: 0x0000000000000011  x22: 0x000000016fe401d0  x23: 0x0000000000000000
   x24: 0x000000010c1fc3d0  x25: 0x0000000000000001  x26: 0x0000000000000001  x27: 0x0000000000000000
   x28: 0x0000000000000000   fp: 0x000000016dabea00   lr: 0x000000016952d9ec
    sp: 0x000000016dabd6e0   pc: 0x00000001026ac008 cpsr: 0x20001000
   far: 0x00000001026ac008  esr: 0x8200000f (Instruction Abort) Permission fault

Binary Images:
               0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???
       0x1028dc000 -        0x102aeffff libjulia-internal.1.8.dylib (*) <1a1874eb-ab80-3290-a5ba-3fcff04a5667> /private/tmp/*/libjulia-internal.1.8.dylib
       0x102338000 -        0x10233bfff julia (*) <bcac4e9b-3685-32c1-9388-7ecf5e1018c0> /private/tmp/*/julia
       0x102448000 -        0x1024a7fff dyld (*) <fbb89662-e6f2-3434-b542-f75185ac5e74> /usr/lib/dyld
       0x1ae107000 -        0x1ae13efff libsystem_kernel.dylib (*) <1d7b3b8e-75a1-34ea-aa52-9f7c23155c55> /usr/lib/system/libsystem_kernel.dylib
       0x1ae13f000 -        0x1ae14bfff libsystem_pthread.dylib (*) <cee8bc77-6923-34d9-89a3-6f8f7279605e> /usr/lib/system/libsystem_pthread.dylib
       0x1ae18e000 -        0x1ae6d3fff com.apple.CoreFoundation (6.9) <8e1809ff-f5b1-3d9e-be15-f2e7b1d8ce51> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
       0x1c0f02000 -        0x1c0f44fff com.apple.DebugSymbols (195.1) <e67e9a02-ffef-3cbc-815b-a5be6e190595> /System/Library/PrivateFrameworks/DebugSymbols.framework/Versions/A/DebugSymbols

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 1
    thread_create: 0
    thread_set_state: 12

VM Region Summary:
ReadOnly portion of Libraries: Total=752.4M resident=0K(0%) swapped_out_or_unallocated=752.4M(100%)
Writable regions: Total=2.6G written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=2.6G(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Activity Tracing                   256K        1 
Kernel Alloc Once                   32K        1 
MALLOC                           736.1M       55 
MALLOC guard page                   96K        5 
MALLOC_LARGE (reserved)           76.3M        1         reserved VM address space (unallocated)
MALLOC_MEDIUM (reserved)         920.0M        8         reserved VM address space (unallocated)
MALLOC_NANO (reserved)           128.0M        1         reserved VM address space (unallocated)
Mach message                        32K        2 
STACK GUARD                         96K        6 
Stack                             66.6M        6 
VM_ALLOCATE                      672.0M    14769 
VM_ALLOCATE (reserved)           260.0M        5         reserved VM address space (unallocated)
__AUTH                             242K       52 
__AUTH_CONST                      2986K      133 
__DATA                            4961K      160 
__DATA_CONST                      7250K      170 
__DATA_DIRTY                       241K       52 
__LINKEDIT                       604.4M       40 
__OBJC_CONST                       279K       31 
__OBJC_RO                         82.9M        1 
__OBJC_RW                         3152K        1 
__TEXT                           148.1M      179 
__UNICODE                          592K        1 
dyld private memory               1024K        1 
mapped file                        624K        2 
shared memory                     1136K        7 
===========                     =======  ======= 
TOTAL                              3.6G    15690 
TOTAL, minus reserved VM space     2.3G    15690 



-----------
Full Report
-----------

{"app_name":"julia","timestamp":"2022-05-04 22:42:26.00 +0200","app_version":"","slice_uuid":"bcac4e9b-3685-32c1-9388-7ecf5e1018c0","build_version":"","platform":1,"share_with_app_devs":0,"is_first_party":1,"bug_type":"309","os_version":"macOS 12.3.1 (21E258)","incident_id":"B1018C98-2828-43B6-8685-2A5572D78B56","name":"julia"}
{
  "uptime" : 150000,
  "procLaunch" : "2022-05-04 22:41:53.3947 +0200",
  "procRole" : "Unspecified",
  "version" : 2,
  "userID" : 502,
  "deployVersion" : 210,
  "modelCode" : "MacBookPro18,3",
  "procStartAbsTime" : 3794539505176,
  "coalitionID" : 550,
  "osVersion" : {
    "train" : "macOS 12.3.1",
    "build" : "21E258",
    "releaseType" : "User"
  },
  "captureTime" : "2022-05-04 22:42:25.8507 +0200",
  "incident" : "B1018C98-2828-43B6-8685-2A5572D78B56",
  "bug_type" : "309",
  "pid" : 23507,
  "procExitAbsTime" : 3795318233888,
  "translated" : false,
  "cpuType" : "ARM-64",
  "procName" : "julia",
  "procPath" : "\/private\/tmp\/*\/julia",
  "parentProc" : "julia",
  "parentPid" : 23500,
  "coalitionName" : "com.apple.Terminal",
  "crashReporterKey" : "2D7F24AB-FBBA-6FE0-0F16-A8B281CB69B5",
  "responsiblePid" : 573,
  "responsibleProc" : "Terminal",
  "wakeTime" : 2328,
  "sleepWakeUUID" : "9CEDEA75-FD5B-4B27-BCCD-57FB53ABAA3D",
  "sip" : "enabled",
  "vmRegionInfo" : "0x1026ac008 is in 0x1026ac000-0x1028ac000;  bytes after start: 8  bytes before end: 2097143\n      REGION TYPE                    START - END         [ VSIZE] PRT\/MAX SHRMOD  REGION DETAIL\n      __LINKEDIT                  1026a4000-1026ac000    [   32K] r--\/rwx SM=COW  ...8proc.2.dylib\n--->  VM_ALLOCATE                 1026ac000-1028ac000    [ 2048K] rw-\/rwx SM=PRV  \n      VM_ALLOCATE                 1028ac000-1028b0000    [   16K] r-x\/rwx SM=COW  ",
  "isCorpse" : 1,
  "exception" : {"codes":"0x0000000000000002, 0x00000001026ac008","rawCodes":[2,4335517704],"type":"EXC_BAD_ACCESS","signal":"SIGBUS","subtype":"KERN_PROTECTION_FAILURE at 0x00000001026ac008"},
  "termination" : {"flags":0,"code":10,"namespace":"SIGNAL","indicator":"Bus error: 10","byProc":"exc handler","byPid":23507},
  "vmregioninfo" : "0x1026ac008 is in 0x1026ac000-0x1028ac000;  bytes after start: 8  bytes before end: 2097143\n      REGION TYPE                    START - END         [ VSIZE] PRT\/MAX SHRMOD  REGION DETAIL\n      __LINKEDIT                  1026a4000-1026ac000    [   32K] r--\/rwx SM=COW  ...8proc.2.dylib\n--->  VM_ALLOCATE                 1026ac000-1028ac000    [ 2048K] rw-\/rwx SM=PRV  \n      VM_ALLOCATE                 1028ac000-1028b0000    [   16K] r-x\/rwx SM=COW  ",
  "extMods" : {"caller":{"thread_create":0,"thread_set_state":0,"task_for_pid":0},"system":{"thread_create":0,"thread_set_state":12,"task_for_pid":1},"targeted":{"thread_create":0,"thread_set_state":0,"task_for_pid":0},"warnings":0},
  "faultingThread" : 0,
  "threads" : [{"triggered":true,"id":11997668,"threadState":{"x":[{"value":6134945928},{"value":4479680624},{"value":0},{"value":0},{"value":18446638525014080208},{"value":32},{"value":2048},{"value":0},{"value":4335517704},{"value":6062006088,"symbolLocation":0,"symbol":"default_repository_create"},{"value":2043},{"value":2045},{"value":19},{"value":4098230451},{"value":4100329472},{"value":194},{"value":33},{"value":1952448512},{"value":0},{"value":6157217040},{"value":4479680624},{"value":17},{"value":6172180944},{"value":0},{"value":4498375632},{"value":1},{"value":1},{"value":0},{"value":0}],"flavor":"ARM_THREAD_STATE64","lr":{"value":6062004716},"cpsr":{"value":536875008},"fp":{"value":6134950400},"sp":{"value":6134945504},"esr":{"value":2181038095,"description":"(Instruction Abort) Permission fault"},"pc":{"value":4335517704,"matchesCrashFrame":1},"far":{"value":4335517704}},"queue":"com.apple.main-thread","frames":[{"imageOffset":4335517704,"imageIndex":0},{"imageOffset":11190846424,"imageIndex":0},{"imageOffset":11191489568,"imageIndex":0},{"imageOffset":11081559496,"imageIndex":0},{"imageOffset":11154099508,"imageIndex":0},{"imageOffset":11085103956,"imageIndex":0},{"imageOffset":11007754428,"imageIndex":0},{"imageOffset":201100,"symbol":"do_call","symbolLocation":188,"imageIndex":1},{"imageOffset":195400,"symbol":"eval_body","symbolLocation":1464,"imageIndex":1},{"imageOffset":196988,"symbol":"jl_interpret_toplevel_thunk","symbolLocation":264,"imageIndex":1},{"imageOffset":294140,"symbol":"jl_toplevel_eval_flex","symbolLocation":4952,"imageIndex":1},{"imageOffset":293848,"symbol":"jl_toplevel_eval_flex","symbolLocation":4660,"imageIndex":1},{"imageOffset":297248,"symbol":"ijl_toplevel_eval_in","symbolLocation":156,"imageIndex":1},{"imageOffset":4809884836,"imageIndex":0},{"imageOffset":4729080108,"imageIndex":0},{"imageOffset":154872,"symbol":"jl_f__call_latest","symbolLocation":76,"imageIndex":1},{"imageOffset":4727606912,"imageIndex":0},{"imageOffset":4727714416,"imageIndex":0},{"imageOffset":4727768496,"imageIndex":0},{"imageOffset":421260,"symbol":"true_main","symbolLocation":188,"imageIndex":1},{"imageOffset":420996,"symbol":"jl_repl_entrypoint","symbolLocation":180,"imageIndex":1},{"imageOffset":16284,"symbol":"main","symbolLocation":12,"imageIndex":2},{"imageOffset":20616,"symbol":"start","symbolLocation":516,"imageIndex":3}]},{"id":11997669,"frames":[{"imageOffset":31808,"symbol":"kevent","symbolLocation":8,"imageIndex":4},{"imageOffset":426736,"symbol":"signal_listener","symbolLocation":196,"imageIndex":1},{"imageOffset":29292,"symbol":"_pthread_start","symbolLocation":148,"imageIndex":5},{"imageOffset":8332,"symbol":"thread_start","symbolLocation":8,"imageIndex":5}]},{"id":11997680,"frames":[{"imageOffset":6352,"symbol":"mach_msg_trap","symbolLocation":8,"imageIndex":4},{"imageOffset":7488,"symbol":"mach_msg","symbolLocation":76,"imageIndex":4},{"imageOffset":38456,"symbol":"mach_msg_server","symbolLocation":320,"imageIndex":4},{"imageOffset":422628,"symbol":"mach_segv_listener","symbolLocation":40,"imageIndex":1},{"imageOffset":29292,"symbol":"_pthread_start","symbolLocation":148,"imageIndex":5},{"imageOffset":8332,"symbol":"thread_start","symbolLocation":8,"imageIndex":5}]},{"id":11997964,"frames":[{"imageOffset":6352,"symbol":"mach_msg_trap","symbolLocation":8,"imageIndex":4},{"imageOffset":7488,"symbol":"mach_msg","symbolLocation":76,"imageIndex":4},{"imageOffset":543772,"symbol":"__CFRunLoopServiceMachPort","symbolLocation":372,"imageIndex":6},{"imageOffset":536780,"symbol":"__CFRunLoopRun","symbolLocation":1180,"imageIndex":6},{"imageOffset":533648,"symbol":"CFRunLoopRunSpecific","symbolLocation":600,"imageIndex":6},{"imageOffset":1111592,"symbol":"CFRunLoopRun","symbolLocation":64,"imageIndex":6},{"imageOffset":21340,"symbol":"SpotlightQueryThread(void*)","symbolLocation":564,"imageIndex":7},{"imageOffset":29292,"symbol":"_pthread_start","symbolLocation":148,"imageIndex":5},{"imageOffset":8332,"symbol":"thread_start","symbolLocation":8,"imageIndex":5}]},{"id":11997975,"frames":[{"imageOffset":8312,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":5}]},{"id":11997977,"frames":[{"imageOffset":8312,"symbol":"start_wqthread","symbolLocation":0,"imageIndex":5}]}],
  "usedImages" : [
  {
    "size" : 0,
    "source" : "A",
    "base" : 0,
    "uuid" : "00000000-0000-0000-0000-000000000000"
  },
  {
    "source" : "P",
    "arch" : "arm64",
    "base" : 4337811456,
    "size" : 2179072,
    "uuid" : "1a1874eb-ab80-3290-a5ba-3fcff04a5667",
    "path" : "\/private\/tmp\/*\/libjulia-internal.1.8.dylib",
    "name" : "libjulia-internal.1.8.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64",
    "base" : 4331896832,
    "size" : 16384,
    "uuid" : "bcac4e9b-3685-32c1-9388-7ecf5e1018c0",
    "path" : "\/private\/tmp\/*\/julia",
    "name" : "julia"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 4333010944,
    "size" : 393216,
    "uuid" : "fbb89662-e6f2-3434-b542-f75185ac5e74",
    "path" : "\/usr\/lib\/dyld",
    "name" : "dyld"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7215280128,
    "size" : 229376,
    "uuid" : "1d7b3b8e-75a1-34ea-aa52-9f7c23155c55",
    "path" : "\/usr\/lib\/system\/libsystem_kernel.dylib",
    "name" : "libsystem_kernel.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7215509504,
    "size" : 53248,
    "uuid" : "cee8bc77-6923-34d9-89a3-6f8f7279605e",
    "path" : "\/usr\/lib\/system\/libsystem_pthread.dylib",
    "name" : "libsystem_pthread.dylib"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7215833088,
    "CFBundleShortVersionString" : "6.9",
    "CFBundleIdentifier" : "com.apple.CoreFoundation",
    "size" : 5529600,
    "uuid" : "8e1809ff-f5b1-3d9e-be15-f2e7b1d8ce51",
    "path" : "\/System\/Library\/Frameworks\/CoreFoundation.framework\/Versions\/A\/CoreFoundation",
    "name" : "CoreFoundation",
    "CFBundleVersion" : "1858.112"
  },
  {
    "source" : "P",
    "arch" : "arm64e",
    "base" : 7531929600,
    "CFBundleShortVersionString" : "195.1",
    "CFBundleIdentifier" : "com.apple.DebugSymbols",
    "size" : 274432,
    "uuid" : "e67e9a02-ffef-3cbc-815b-a5be6e190595",
    "path" : "\/System\/Library\/PrivateFrameworks\/DebugSymbols.framework\/Versions\/A\/DebugSymbols",
    "name" : "DebugSymbols",
    "CFBundleVersion" : "195.1"
  }
],
  "sharedCache" : {
  "base" : 7212220416,
  "size" : 3136438272,
  "uuid" : "1df3dfc1-141a-35d0-a4e5-f1e113894c6e"
},
  "vmSummary" : "ReadOnly portion of Libraries: Total=752.4M resident=0K(0%) swapped_out_or_unallocated=752.4M(100%)\nWritable regions: Total=2.6G written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=2.6G(100%)\n\n                                VIRTUAL   REGION \nREGION TYPE                        SIZE    COUNT (non-coalesced) \n===========                     =======  ======= \nActivity Tracing                   256K        1 \nKernel Alloc Once                   32K        1 \nMALLOC                           736.1M       55 \nMALLOC guard page                   96K        5 \nMALLOC_LARGE (reserved)           76.3M        1         reserved VM address space (unallocated)\nMALLOC_MEDIUM (reserved)         920.0M        8         reserved VM address space (unallocated)\nMALLOC_NANO (reserved)           128.0M        1         reserved VM address space (unallocated)\nMach message                        32K        2 \nSTACK GUARD                         96K        6 \nStack                             66.6M        6 \nVM_ALLOCATE                      672.0M    14769 \nVM_ALLOCATE (reserved)           260.0M        5         reserved VM address space (unallocated)\n__AUTH                             242K       52 \n__AUTH_CONST                      2986K      133 \n__DATA                            4961K      160 \n__DATA_CONST                      7250K      170 \n__DATA_DIRTY                       241K       52 \n__LINKEDIT                       604.4M       40 \n__OBJC_CONST                       279K       31 \n__OBJC_RO                         82.9M        1 \n__OBJC_RW                         3152K        1 \n__TEXT                           148.1M      179 \n__UNICODE                          592K        1 \ndyld private memory               1024K        1 \nmapped file                        624K        2 \nshared memory                     1136K        7 \n===========                     =======  ======= \nTOTAL                              3.6G    15690 \nTOTAL, minus reserved VM space     2.3G    15690 \n",
  "legacyInfo" : {
  "threadTriggered" : {
    "queue" : "com.apple.main-thread"
  }
},
  "trialInfo" : {
  "rollouts" : [
    {
      "rolloutId" : "5fc94383418129005b4e9ae0",
      "factorPackIds" : {

      },
      "deploymentId" : 240000368
    },
    {
      "rolloutId" : "60356660bbe37970735c5624",
      "factorPackIds" : {

      },
      "deploymentId" : 240000027
    }
  ],
  "experiments" : [

  ]
}
}

@KristofferC
Copy link
Member

It does get stuck a bit later:

You can change

julia/contrib/generate_precompile.jl

Line 245 in 862018b

debug_output = devnull # or stdout
to stdout to see if the print out reveals anything.

@fxcoudert
Copy link
Contributor Author

@KristofferC

Generating REPL precompile statements... 25/40
#### inputting statement: ####
"Pkg.add(\"TestPkg\")"
####
julia> Pkg.add("TestPkg")
   Resolving package versions...
     Cloning [b102fd66-69d9-46bc-b342-4864833e9d5c] TestPkg from /private/tmp/jl_9WoRnc/TestPkg.jl

signal (10): Bus error: 10
in expression starting at REPL[16]:1
unknown function (ip: 0x100744008)
Allocations: 114365026 (Pool: 114309879; Big: 55147); GC: 112

@fxcoudert
Copy link
Contributor Author

Under lldb I get a slightly better backtrace:

  * frame #0: 0x0000000100600008
    frame #1: 0x000000015e6099ec libgit2.1.4.3.dylib`git__clone + 240
    frame #2: 0x000000028dbfc108
    frame #3: 0x00000002863a96c0
    frame #4: 0x0000000286b24478
    frame #5: 0x000000028694c21c
    frame #6: 0x0000000169da4088
    frame #7: 0x000000010103118c libjulia-internal.1.8.dylib`do_call + 188
    frame #8: 0x000000010102fb48 libjulia-internal.1.8.dylib`eval_body + 1464
    frame #9: 0x000000010103017c libjulia-internal.1.8.dylib`jl_interpret_toplevel_thunk + 264
    frame #10: 0x0000000101047cfc libjulia-internal.1.8.dylib`jl_toplevel_eval_flex + 4952
    frame #11: 0x0000000101047bd8 libjulia-internal.1.8.dylib`jl_toplevel_eval_flex + 4660
    frame #12: 0x0000000101048920 libjulia-internal.1.8.dylib`ijl_toplevel_eval_in + 156
    frame #13: 0x000000015c3c8480
    frame #14: 0x000000015a46864c
    frame #15: 0x0000000101025cf8 libjulia-internal.1.8.dylib`jl_f__call_latest + 76
    frame #16: 0x000000015a434664
    frame #17: 0x0000000101066d8c libjulia-internal.1.8.dylib`true_main + 188
    frame #18: 0x0000000101066c84 libjulia-internal.1.8.dylib`jl_repl_entrypoint + 180
    frame #19: 0x0000000100003f9c julia`main + 12
    frame #20: 0x0000000100011088 dyld`start + 516

We're building against system libgit2, so maybe that's another non upstreamed patch.

@fxcoudert
Copy link
Contributor Author

I hoped this dependency would allow to install llvm before llvmunwind is configured:

ifeq ($(USE_SYSTEM_LLVM),0)
 $(BUILDDIR)/llvmunwind-$(LLVMUNWIND_VER)/build-configured: | $(LLVM_BUILDDIR_withtype)/build-compiled
 endif

but it does not work reliably: it builds llvmunwind after llvm is built, but not necessarily installed (so llvm-config is not present). Can someone suggest a way to add the dependency to installed llvm? I cannot figure it out.

@fxcoudert
Copy link
Contributor Author

We're building against system libgit2, so maybe that's another non upstreamed patch.

I used the exact same build, but without the system libgit2 (removed USE_SYSTEM_LIBGIT2=1). Then, the build fails even earlier in the process:

./build_h.jl
error during bootstrap:
LoadError(at "sysimg.jl" line 3: LoadError(at "Base.jl" line 161: LoadError(at "./build_h.jl" line 10: ErrorException("syntax: invalid escape sequence"))))
jl_f_throw at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
ijl_apply_generic at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
do_call at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
eval_body at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_interpret_toplevel_thunk at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
top-level scope at ./build_h.jl:10
jl_toplevel_eval_flex at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_eval_errorf at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_toplevel_eval_flex at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_parse_eval_all at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
ijl_load_ at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
include at ./boot.jl:366 [inlined]
include at ./Base.jl:15
include at ./Base.jl:17
unknown function (ip: 0x10e3d4097)
ijl_apply_generic at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
do_call at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
eval_body at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_interpret_toplevel_thunk at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
top-level scope at Base.jl:161
jl_toplevel_eval_flex at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_toplevel_eval_flex at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_parse_eval_all at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
ijl_load_ at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
unknown function (ip: 0x10e24806b)
include at ./boot.jl:366
unknown function (ip: 0x10e2540bb)
ijl_apply_generic at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
do_call at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
eval_body at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_interpret_toplevel_thunk at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
top-level scope at sysimg.jl:3
jl_toplevel_eval_flex at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_parse_eval_all at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
ijl_load_ at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
ijl_load at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
exec_program at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
true_main at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)
jl_repl_entrypoint at /private/tmp/julia-20220505-16215-1a60ogs/julia-1.8.0-beta3/usr/lib/libjulia-internal.1.8.dylib (unknown line)

One step forward, two steps back.

@fxcoudert fxcoudert mentioned this pull request May 5, 2022
Closed
@fxcoudert
Copy link
Contributor Author

OK, I've now pushed a clean PR that fixes the missing patches issue, and allows compilation to proceed further. It does not fix problems that arise later, but it should be safe to commit and fix one issue at a time…

@giordano could you review, or indicate who has review/commit privileges for the build system?

@KristofferC KristofferC mentioned this pull request May 16, 2022
Merged
67 tasks
@giordano giordano merged commit f18324c into JuliaLang:master May 17, 2022
@giordano giordano mentioned this pull request May 17, 2022
Closed
@fxcoudert fxcoudert deleted the patches branch May 17, 2022 21:58
KristofferC pushed a commit that referenced this pull request May 18, 2022
@fxcoudert @KristofferC
Resurrect libunwind patches (#45189)
123778d 
Fixes #44499

(cherry picked from commit f18324c)
KristofferC pushed a commit that referenced this pull request May 18, 2022
@fxcoudert @KristofferC
Resurrect libunwind patches (#45189)
43ba8ae 
Fixes #44499

(cherry picked from commit f18324c)
@fxcoudert fxcoudert mentioned this pull request May 19, 2022
Closed
@KristofferC KristofferC removed the backport 1.8 Change should be backported to release-1.8 label May 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vchuravy vchuravy vchuravy approved these changes

Assignees
No one assigned
Labels
building Build system, or building Julia or its dependencies external dependencies Involves LLVM, OpenBLAS, or other linked libraries system:mac Affects only macOS
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Build failure due to missing patch llvm-libunwind-prologue-epilogue.patch
4 participants
@fxcoudert @giordano @KristofferC @vchuravy