Latest API (3.12) for macOS build issues

[lagomorph]

API 3.12 puts include files in /usr/local/inc. I had to add /usr/local/inc to FindLibSDRplay.cmake to get cmake to find the API files.
In addition, the library file that gets built (libsdrPlaySupport.so) has a shared library libsdrplay_api_x86_64.so.3.12 that doesn't match the API's libsdrplay_api.so.3.12.0. This causes a runtime error:

Module found: /usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so
dlopen() failed: dlopen(/usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so, 0x0001): Library not loaded: libsdrplay_api_x86_64.so.3.12
Referenced from: /usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so
Reason: tried: 'libsdrplay_api_x86_64.so.3.12' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibsdrplay_api_x86_64.so.3.12' (no such file), 'libsdrplay_api_x86_64.so.3.12' (no such file), '/Users/xxx/Documents/Development/git/SoapySDRPlay3/build/libsdrplay_api_x86_64.so.3.12' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/xxx/Documents/Development/git/SoapySDRPlay3/build/libsdrplay_api_x86_64.so.3.12' (no such file), '/Users/xxx/Documents/Development/git/SoapySDRPlay3/build/libsdrplay_api_x86_64.so.3.12' (no such file)

I don't have a workaround for this other than creating a symlink everywhere I need to use SoapySDRPlay.
ln -s /Library/SDRplayAPI/3.12.0/lib/libsdrplay_api.so.3.12.0 libsdrplay_api_x86_64.so.3.12

Any ideas where this comes from?

[SDRplay]

The Mac API installer has a few issues. I'm in the middle of sorting it out and I'll post here when it's released. It's a universal library so there are extra steps needed to get the libraries visible which I've been working on today.

[fventuri]

Looks like the Mac API installer for version 3.12 is fixed now: https://groups.io/g/SDRPlayUsers/message/9046
Thanks Andy for all your work!

Franco

[SDRplay]

Hi Franco, yes, finished testing it with Steve at about 4am this morning, grabbed a few hours kip, being forced to go Christmas shopping lol and then later I'll sort out API 3.12 for the other platforms - any problems I'm sure you'll all let me know! This version of the API installer should tidy up the mess caused by the previous one so you should just be able to run it.

[lagomorph]

Thank you for all your work! For some reason I still need the symlink wherever I want to use SoapySDR.
ln -s /usr/local/lib/libsdrplay_api.so.3.12 . otherwise I get:

[ERROR] SoapySDR::loadModule(/usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so) dlopen() failed: dlopen(/usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so, 0x0001): Library not loaded: libsdrplay_api.so.3.12 Referenced from: <CB0D967C-BDE5-35C3-A6ED-1B35863E361E> /usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so Reason: tried: 'libsdrplay_api.so.3.12' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibsdrplay_api.so.3.12' (no such file), 'libsdrplay_api.so.3.12' (no such file), '/Users/xxx/Library/Mobile Documents/com~apple~CloudDocs/libsdrplay_api.so.3.12' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/xxx/Library/Mobile Documents/com~apple~CloudDocs/libsdrplay_api.so.3.12' (no such file), '/Users/xxx/Library/Mobile Documents/com~apple~CloudDocs/libsdrplay_api.so.3.12' (no such file)

This is macOS Sonoma.

Also, are the docs for 3.12 somewhere? The install program has a link of https://www.sdrplay.com/docs/SDRplay_API_Specification_v3.12.pdf but that's a 404. On your website the API docs link to https://www.sdrplay.com/docs/SDRplay_API_Specification_v3.09.pdf.

[SDRplay]

Hi, I'm still editing the 3.12 spec so it will be up asap - there's not much different to the 3.09 so you're good with that one for now.

Regarding the library, all of the API libraries go into /usr/local/lib and that should be on your library search path. It looks like on your system it's looking in /Users/xxx/Library/Mobile Documents/comappleCloudDocs - I've no idea why - I built SoapySDR and SoapySDRPlay on a mac after installing API 3.12 last night and it all built and ran by just doing the following...

mkdir -p ~/Dev
cd ~/Dev
git clone https://github.com/Pothosware/SoapySDR
cd SoapySDR
mkdir build
cd build
cmake ..
make
sudo make install

cd ~/Dev
git clone https://github.com/Pothosware/SoapySDRPlay
cd SoapySDRPlay
mkdir build
cd build
cmake ..
make
sudo make install

the API was found and used without a problem.

Hope that helps.

Andy

[lagomorph]

/Users/xxx/Library/Mobile Documents/comappleCloudDocs/ just happened to be my current directory

Those are my same build steps and I do get the libs in /usr/local/lib

$ ls -l /usr/local/lib/libsdrplay_api*
lrwxr-xr-x 1 root admin 32 Dec 2 07:03 /usr/local/lib/libsdrplay_api.dylib -> /usr/local/lib/libsdrplay_api.so
lrwxr-xr-x 1 root admin 34 Dec 2 07:03 /usr/local/lib/libsdrplay_api.so -> /usr/local/lib/libsdrplay_api.so.3
lrwxr-xr-x 1 root admin 37 Dec 2 07:03 /usr/local/lib/libsdrplay_api.so.3 -> /usr/local/lib/libsdrplay_api.so.3.12
lrwxr-xr-x 1 root admin 53 Dec 2 07:03 /usr/local/lib/libsdrplay_api.so.3.12 -> /Library/SDRplayAPI/3.12.1/lib/libsdrplay_api.so.3.12
lrwxr-xr-x 1 root admin 32 Dec 2 07:03 /usr/local/lib/libsdrplay_api_x86_64.so.3.12 -> /usr/local/lib/libsdrplay_api.so

DYLD_LIBRARY_PATH=/usr/local/lib SoapySDRUtil --info is another workaround but I don't have issues with other libs in /usr/local/lib.

[fventuri]

@lagomorph - I do not have a Mac, but I think these two commands might be useful in troubleshooting your problem:

otool -L  /usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so | grep sdrplay_api
otool -l  /usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so | grep sdrplay_api

(the first one has uppercase '-L', the second one instead has lowercase '-l').

Franco

[lagomorph]

@fventuri Thanks for the suggestion. I was using otool -L earlier to make sure the name of the library in the API didn't have the x86_64 in it anymore (which it doesn't) but looking closer at it I see that the dynamic lib for the API doesn't include the path to /usr/local/lib. I added the path with install_name_tool -change libsdrplay_api.so.3.12 /usr/local/lib/libsdrplay_api.so.3.12 /usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so and it loads fine now. Maybe some people have /usr/local/lib in DYLD_LIBRARY_PATH but mine is empty.

So bottom line it's fixed for me. Maybe it would be better if the path was explicit in libsdrPlaySupport.so since it's known where it should be.

[fventuri]

@lagomorph - glad to hear it is fixed for you.

I did some quick research on your issue and I found this interesting page in the CMake Wiki: https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling

If I understand correctly, the problem you are experiencing is very similar to the one described there in the 'Common questions' section, so I just created a new branch called install-rpath, where I just added the line they suggest to the CMakeLists.txt file: https://github.com/pothosware/SoapySDRPlay3/blob/install-rpath/CMakeLists.txt#L55

Please give it a try, and let me know how it goes.

Franco

[lagomorph]

@fventuri unfortunately it didn't seem to change anything. I still get a libsdrplay_api.so.3.12 with no path. I also tried all the other examples in the "Always full RPATH" section but I couldn't find any combination that set the path in the library. They do mention in the Caveats section: "Since install-side RPATH tweaking is an operation that is done by target-specific installation handling, any target that should have its install RPATH changed (e.g. to CMAKE_INSTALL_RPATH) needs to end up in the installation via an install(TARGETS ...) signature and not via directory-based "copying" (otherwise the project's cmake_install.cmake file will end up without any RPATH lines)." I'm not sure if that applies.

[fventuri]

@lagomorph - I am not really sure why RPATH is not working there.

Please try rebuilding it with VERBOSE=1, like this:

cmake ..
make VERBOSE=1

The last few lines should show you exactly how the SoapySDRPlay module gets linked.
For instance here on Fedora Linux, using the original CMakeLists.txt from the master branch, this is what I see:

[100%] Linking CXX shared module libsdrPlaySupport.so
/usr/bin/cmake -E cmake_link_script CMakeFiles/sdrPlaySupport.dir/link.txt --verbose=1
/usr/lib64/ccache/c++ -fPIC -O3 -DNDEBUG -pthread -shared  -o libsdrPlaySupport.so CMakeFiles/sdrPlaySupport.dir/Registration.cpp.o CMakeFiles/sdrPlaySupport.dir/sdrplay_api.cpp.o CMakeFiles/sdrPlaySupport.dir/Settings.cpp.o CMakeFiles/sdrPlaySupport.dir/Streaming.cpp.o CMakeFiles/sdrPlaySupport.dir/Version.cpp.o  -Wl,-rpath,/usr/local/lib64: /usr/local/lib64/libsdrplay_api.so /usr/local/lib64/libSoapySDR.so.0.8.1 -Wl,--no-undefined -pthread 

In my case the path to the local shared libraries is /usr/local/lib64 (instead of /usr/local/lib) because of where Fedora stores the 64 bit shared libraries, however notice the -Wl,-rpath,/usr/local/lib64:, which I think is what sets the real time search path in my libsdrPlaySupport.so shared library.

I wonder if on your Mac you see something similar or not.

Franco

[lagomorph]

Yes, I see something similar. There are two different -Wl arguments though. I wonder if that has anything to do with it.

[100%] Linking CXX shared module libsdrPlaySupport.so /usr/local/Cellar/cmake/3.27.8/bin/cmake -E cmake_link_script CMakeFiles/sdrPlaySupport.dir/link.txt --verbose=1 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -O3 -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -bundle -Wl,-headerpad_max_install_names -o libsdrPlaySupport.so CMakeFiles/sdrPlaySupport.dir/Registration.cpp.o CMakeFiles/sdrPlaySupport.dir/sdrplay_api.cpp.o CMakeFiles/sdrPlaySupport.dir/Settings.cpp.o CMakeFiles/sdrPlaySupport.dir/Streaming.cpp.o CMakeFiles/sdrPlaySupport.dir/Version.cpp.o -Wl,-rpath,/usr/local/lib /usr/local/lib/libsdrplay_api.dylib /usr/local/lib/libSoapySDR.0.8.1.dylib -pthread -flat_namespace
[100%] Built target sdrPlaySupport /usr/local/Cellar/cmake/3.27.8/bin/cmake -E cmake_progress_start /Users/xxx/Documents/Development/git/SoapySDRPlay3/build/CMakeFiles 0

$ otool -L libsdrPlaySupport.so
libsdrPlaySupport.so: libsdrplay_api.so.3.12 (compatibility version 0.0.0, current version 0.0.0) /usr/local/opt/soapysdr/lib/libSoapySDR.0.8.dylib (compatibility version 0.8.0, current version 0.8.1) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1600.151.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.0.0)

[ahooper]

@lagomorph Your install_name_tool suggestion was very helpful! Recent MacOS Sonoma updates had broken the libsdrplay_api linkages in my newly built applications. I am guessing this is due to further tightening of dyld security, and wonder if the Apple way is for the @SDRplay package to have the installation path built into the library before code signing Quinn's advice? That might avoid the library-using applications having to patch it up in their builds (e.g. sdrangel).

[SDRplay]

I've had complaints over the years regardless of where I install the library, whether it's usr or local or opt or lib or lib64 - this always seems to upset someone. So I decided the best way was to install the library somewhere and then link to wherever people want it linked to - then you just need to make sure that wherever it's linked to is in LD_LIBRARY_PATH - surely that's all that needs doing?

I had no problems building SoapySDRPlay out of the box so I'm not sure why other applications are having build issues?

I don't think rewriting the library with a different install path and hence breaking its signing/notarizing status is a good idea, but each to their own.

[ahooper]

I definitely understand the decision to choose one place, and am not advocating otherwise. If you're running Sonoma 14.2 and Xcode 15 try this little test program:

$ cat sdrplay_api_version.c
#include <stdio.h>
#include <sdrplay_api.h>

int main(int argc, const char * argv[]) {
    float version = 0;
    sdrplay_api_Open();
    sdrplay_api_ApiVersion(&version);
    printf("sdrplay_api_ApiVersion %f \n", version);
    sdrplay_api_Close();
    return 0;
}

$ cc -o sdrplay_api_version sdrplay_api_version.c -lsdrplay_api
$ ./sdrplay_api_version
dyld[84908]: Library not loaded: libsdrplay_api.so.3.12
  Referenced from: <0BFE4439-92E1-3418-B011-EE3B3339F61A> /Users/andy/Software/Xcode/try/sdrplay/sdrplay_api_version
  Reason: tried: 'libsdrplay_api.so.3.12' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibsdrplay_api.so.3.12' (no such file), 'libsdrplay_api.so.3.12' (no such file), '/Users/andy/Software/Xcode/try/sdrplay/libsdrplay_api.so.3.12' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/andy/Software/Xcode/try/sdrplay/libsdrplay_api.so.3.12' (no such file), '/Users/andy/Software/Xcode/try/sdrplay/libsdrplay_api.so.3.12' (no such file)
Abort trap: 6
$ install_name_tool -change libsdrplay_api.so.3.12 /usr/local/lib/libsdrplay_api.so.3.12 sdrplay_api_version
$ ./sdrplay_api_version
sdrplay_api_ApiVersion 3.120000

I have been struggling for several days to understand this. My best guess is that dyld is no longer looking at /usr/local/lib, even if I include -L/usr/local/lib on the cc, or give the full path of the .so, and suspect Apple wants the library LC_ID_DYLIB to include a relative prefix such @rpath

[SDRplay]

I'm building on MacOS 13.4.1 (Apple M1) and Xcode is version 14.3.1 - I don't want to mess with that because this is the SDRconnect build engine.

I've just built and run that code without any warnings or errors, so is this something that's come into a later MacOS or Xcode version?

[ahooper]

Yes, on Apple M2 when I moved from MacOS 13.something to 14.1. I'd been happily using sdrplay_api for several applications on earlier MacOS. Don't mess with SDRconnect :-) I will continue looking for clues.

[SDRplay]

I just logged into a colleague's Mac... MacOS 14.1.2 (Apple Silicon) with Xcode 15.1...

I see the library not found error at runtime, but setting DYLD_LIBRARY_PATH fixed the problem...

$ DYLD_LIBRARY_PATH=/usr/local/lib ./sdrplay_api_version
sdrplay_api_ApiVersion 3.120000

I guess I could have also...
$ export DYLD_LIBRARY_PATH=/usr/local/lib
$ ./sdrplay_api_version

but I didn't try that. Does that work for you? I'm not sure why DYLD_LIBRARY_PATH is handled differently between the version I'm on and the latest?

Andy.

[SDRplay]

In the good old days under Linux you would run "sudo ldconfig" and it would re-catalog the paths that needed to be searched for libraries, but I don't think MacOS has an equivalent of that?

[ahooper]

Yes, DYLD environment variables will hide it, but that seems hackey and is blocked by the default Hardened Runtime. The best I can tell, the standard way to build a movable library is to use the linker option -install_name @rpath/libsdrplay_api.so. See https://www.mikeash.com/pyblog/friday-qa-2009-11-06-linking-and-install-names.html, https://cmake.org/cmake/help/latest/prop_tgt/MACOSX_RPATH.html, http://clarkkromenaker.com/post/library-dynamic-loading-mac

[fventuri]

@ahooper - a couple of weeks ago @lagomorph tried with various RPATH settings on his Mac (see his comment here: #79 (comment)), but I don't think he was successful.

I do not have a Mac that I can use here, so I can't really help you much, but if you do find a good cmake configuration that addresses this problem on the newer version of MacOS (without affecting older versions), I'll be happy to add it to the CMakeLists.txt configuration file in this repository.

Franco

[lagomorph]

Yes, I wasn't able to get RPATH settings to have any effect. Something like:

if (APPLE)
    add_custom_command(TARGET sdrPlaySupport
        POST_BUILD COMMAND
        ${CMAKE_INSTALL_NAME_TOOL} -change libsdrplay_api.so.3.12 /usr/local/lib/libsdrplay_api.so.3.12 libsdrPlaySupport.so)
endif()

works but that seems hacky.

[ahooper]

Here is a revision of @lagomorph's suggestion, to be independent of libsdrplay_api version:

if(APPLE)
    # libsdrplay_api.so install path required by macOS 14 (Sonoma)
    execute_process(COMMAND readlink -f ${LIBSDRPLAY_LIBRARIES}
        	OUTPUT_STRIP_TRAILING_WHITESPACE
        	OUTPUT_VARIABLE LIBSDRPLAY_INSTNAME)
    execute_process(COMMAND basename "${LIBSDRPLAY_INSTNAME}"
        	OUTPUT_STRIP_TRAILING_WHITESPACE
        	OUTPUT_VARIABLE LIBSDRPLAY_DEPNAME)
    add_custom_command(TARGET sdrPlaySupport
    	POST_BUILD COMMAND
    	${CMAKE_INSTALL_NAME_TOOL} -change ${LIBSDRPLAY_DEPNAME} ${LIBSDRPLAY_INSTNAME} $<TARGET_FILE_NAME:sdrPlaySupport>)
endif(APPLE)

This could be eliminated if @SDRplay could change the build of the library to add the -install_name option to the .so, set to the location the .pkg installs it to, i.e. -install_name /Library/SDRplayAPI/3.14.0/lib/libsdrplay_api.so.3.14.

man ld
. . .
   Options when creating a dynamic library (dylib)
     -install_name name
             Sets an internal "install path" (LC_ID_DYLIB) in a dynamic
             library. Any clients linked against the library will record that
             path as the way dyld should locate this library.  If this option
             is not specified, then the -o path will be used.  This option is
             also called -dylib_install_name for compatibility.
. . .

dyld no longer has a fallback library search path to /usr/local/lib:/usr/lib, so a library dependency without a directory path will only be found in the current working directory or in the macOS /System Cryptexes.

man dyld
. . .
DYNAMIC LIBRARY LOADING
       Unlike many other operating systems, Darwin does not locate dependent
       dynamic libraries via their leaf file name.  Instead the full path to
       each dylib is used (e.g. /usr/lib/libSystem.B.dylib).  But there are
       times when a full path is not appropriate; for instance, may want your
       binaries to be installable in anywhere on the disk.  To support that,
       there are three @xxx/ variables that can be used as a path prefix.  At
       runtime dyld substitutes a dynamically generated path for the @xxx/
       prefix.
. . .

[fventuri]

@ahooper - thanks for your suggestion.

If you were to build your sdrplay_api_version example from December with a command like this:

cc -o sdrplay_api_version sdrplay_api_version.c /usr/local/lib/libsdrplay_api.so

(instead of -lsdrplay_api) would that hardcode the full path of the SDRplay API shared object, thus removing the need of running install_name_tool -change afterwards?

Franco

[ahooper]

No, the library path built into the binary is the one explicitly set in the .so library:

$ cc -o sdrplay_api_version sdrplay_api_version.c /usr/local/lib/libsdrplay_api.so
$ ./sdrplay_api_version
dyld[12438]: Library not loaded: libsdrplay_api.so.3.14
  Referenced from: <15B435EC-1C0F-36F7-B2C0-EE76E290BAE5> /Users/andy/Software/Xcode/try/sdrplay/sdrplay_api_version
  Reason: tried: 'libsdrplay_api.so.3.14' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibsdrplay_api.so.3.14' (no such file), 'libsdrplay_api.so.3.14' (no such file), '/Users/andy/Software/Xcode/try/sdrplay/libsdrplay_api.so.3.14' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/andy/Software/Xcode/try/sdrplay/libsdrplay_api.so.3.14' (no such file), '/Users/andy/Software/Xcode/try/sdrplay/libsdrplay_api.so.3.14' (no such file)
Abort trap: 6
$ install_name_tool -change libsdrplay_api.so.3.14 /Library/SDRplayAPI/3.14.0/lib/libsdrplay_api.so.3.14 sdrplay_api_version
$ ./sdrplay_api_version
sdrplay_api_ApiVersion 3.140000
$ readlink -f /usr/local/lib/libsdrplay_api.so
/Library/SDRplayAPI/3.14.0/lib/libsdrplay_api.so.3.14
$ otool -l /Library/SDRplayAPI/3.14.0/lib/libsdrplay_api.so.3.14
. . .
Load command 4
          cmd LC_ID_DYLIB
      cmdsize 48
         name libsdrplay_api.so.3.14 (offset 24)
. . .

[fventuri]

@lagomorph @ahooper - since I don't have a Mac here, today I spent some time using GitHub CI that can run on MacOS Sonoma workers.

After a few tests with @ahopper simple test program (see here: https://github.com/fventuri/mac-test/actions), I figured out the following:

• the runtime path in a shared library or dynamically linked object is stored in load commands called LC_RPATH, and you can see them with something like this otool -l <filename> | grep -B1 -A4 LC_RPATH
• since the MacOS installer for the SDRplay API copies the shared library libsdrplay_api.so.3 under '/usr/local/lib' (it is actually a symbolic link but that doesn't really matter), the first thing to do is to make sure the runtime path above contains '/usr/local/lib'
• in GNU Radio it looks like the runtime path already contains '/usr/local/lib' (see here: https://github.com/fventuri/gr-sdrplay3/actions), but with SoapySDR (that I installed in the GitHub CI Mac worker using homebrew) it doesn't (see here: https://github.com/pothosware/SoapySDRPlay3/actions)
• long story short: we need to add '/usr/local/lib' to the runtime path but only in the case of this SoapySDRPlay3 module
• the second thing I discovered today is that the MacOS dynamic linker (dynlib) looks for a library in the runtime path only if the install library name starts with the prefix @rpath/, i.e. if the install library name is just libsdrplay_api.so.3 the dynamic linker won't be able to find the library, but if it is @rpath/libsdrplay_api.so.3 it will find it (see my initial tests with @ahopper 's simple example)
• you can see the install name of the SDRplay API library inside the shared libraries built in GNU Radio gr-sdrplay3 and in this SoapySDRPlay3 module with the command otool -L <filename>
• since SDRplay used just libsdrplay_api.so.3 as the library install name when they built that library, then we need to change it in both GNU Radio gr-sdrplay3 libraries and in the libsdrPlaySupport.so library in this SoapySDRPlay3 module to @rpath/libsdrplay_api.so.3
• since both these tasks can be done using install_name_tool (the first one with -add_rpath, the second one with -change), I added these commands to the cmake configuration files CMakeLists.txt
• after the install_name_tool commands you can see how the install name of the SDRplay API library and the runtime path have been changed by running the commands mentioned above: otool -L <shared library filename> and otool -l <shared library filename> | grep -B1 -A4 LC_RPATH
• you'll find my changes in the fix-xcode branch for this SoapySDRPlay3 module: https://github.com/pothosware/SoapySDRPlay3/tree/fix-xcode and in the fx-code branch for the gr-sdrplay3 GNU Radio module: https://github.com/fventuri/gr-sdrplay3/tree/fix-xcode

Franco