Can no longer find debug symbols after macOS 15 sequoia upgrade

[byroot]

My knowledge about linkers and such is extremely limited, so I'm afraid I won't be able to give a very detailed bug report.

I'm an happy use of samply on macOS, and I've been using it a lot to profile native Ruby gems, such as https://github.com/ruby/json. However, since I upgraded my machine to macOS 15 (Sequoia), it's no longer able to find symbols for the compiled bundle, so it makes the profiler pretty much unusable.

Since I have another machine still on macOS 14, I did look at possible differences. On the old machine when recording with --verbose I see two files being opened that aren't on macOS 15:

Opening file "/Users/byroot/src/github.com/ruby/json/ext/json/ext/generator.bundle"
Opening file "/Users/byroot/src/github.com/ruby/json/ext/json/ext/generator/generator.o"

From some googling around, it seems like Apple made some change to how they generate/store debug symbols, now you're supposed to read them from a .bundle.dSYM directory?

Looking at the files generated by clang, it looks like this:

./ext/json/ext/generator.bundle
./ext/json/ext/generator/generator.o
./ext/json/ext/generator/generator.bundle.dSYM
./ext/json/ext/generator/generator.bundle.dSYM/Contents
./ext/json/ext/generator/generator.bundle.dSYM/Contents/Resources
./ext/json/ext/generator/generator.bundle.dSYM/Contents/Resources/Relocations
./ext/json/ext/generator/generator.bundle.dSYM/Contents/Resources/Relocations/aarch64
./ext/json/ext/generator/generator.bundle.dSYM/Contents/Resources/Relocations/aarch64/generator.bundle.yml
./ext/json/ext/generator/generator.bundle.dSYM/Contents/Resources/DWARF
./ext/json/ext/generator/generator.bundle.dSYM/Contents/Resources/DWARF/generator.bundle
./ext/json/ext/generator/generator.bundle.dSYM/Contents/Info.plist

The .o still exists, but samply no longer tries to open it.

[mstange]

I can reproduce this locally. Example profile: https://share.firefox.dev/3YtKZuF

I think the problem is on the sampling side, not on the symbolication side. Looking into it.

[mstange]

If I attach to the process with lldb, it has these five modules at the end of image list:

[299] 4FA2B384-8C96-3984-B984-A8461E6096AD 0x0000000100134000 /opt/homebrew/Cellar/ruby/3.3.5/lib/ruby/3.3.0/arm64-darwin24/enc/encdb.bundle 
[300] A0818C55-9967-3749-B96E-4F1F3FAD7F82 0x0000000100160000 /opt/homebrew/Cellar/ruby/3.3.5/lib/ruby/3.3.0/arm64-darwin24/enc/trans/transdb.bundle 
[301] 4E8F8FC3-B7BE-30EC-8E63-097984739A20 0x0000000100450000 /opt/homebrew/Cellar/ruby/3.3.5/lib/ruby/3.3.0/arm64-darwin24/monitor.bundle 
[302] 8F171032-37BE-3928-88A6-23AECBDEE568 0x00000001000a0000 /opt/homebrew/lib/ruby/gems/3.3.0/gems/json-2.7.2/lib/json/ext/parser.bundle 
[303] 0D550828-F949-3239-B400-3F9D82D87934 0x0000000100480000 /opt/homebrew/lib/ruby/gems/3.3.0/gems/json-2.7.2/lib/json/ext/generator.bundle 

However, in my enumerate_dyld_images function, I get 0 for both imageLoadAddress and imageFilePath for these last five modules. I haven't found out why yet.

[Maaarcocr]

it may not be of big help, but I can also reproduce this issue with a much smaller example, just trying to dynamically load a rust cdylib from a rust executable: https://github.com/Maaarcocr/test_samply

I thought maybe ruby was doing something fancy on the new macOS version, but I guess that is not it.

[Maaarcocr]

Ok so i've done some investigations and the main issue seems to stem from here. In fact, if I comment out this line which essentially means remapping memory every time we have to read some of the profiled processed memory, the names return.

Something somehow is making the mapped memory get out of sync. I have made a simple example with [email protected]:Maaarcocr/test-remap.git to check that remap does still work on the new macos version and it does seem to do so.
Maybe apple has changed how that specific part of memory that keeps track of dyld info can be mapped and it can only be copied over? Unsure.

I've confirmed that if I use proc-maps https://github.com/rbspy/proc-maps I can get the names back (they do not use the mapped memory approach that samply uses). Now I'm not sure what the perf implications of doing that are, but I would take samply being a bit slower versus samply not displaying symbols for dlopen-ed libraries.