symbol(s) not found for architecture x86_64 (macOS)

[edwardloveall]

When building invidious with crystal build src/invidious.cr --release I get this error:

$ crystal build src/invidious.cr --release 
crystal build src/invidious.cr --release 
ld: warning: ignoring file /path/to/invidious/lib/lsquic/src/lsquic/ext/liblsquic.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )
Undefined symbols for architecture x86_64:
  "_SSL_CTX_set_mode", referenced from:
      _*OpenSSL::SSL::Context::Client::new:OpenSSL::SSL::Context::Client in _main.o
      _*Kemal::CLI::new<Array(String)>:Kemal::CLI in _main.o
  "_SSL_CTX_set_options", referenced from:
      _*OpenSSL::SSL::Context::Client::new:OpenSSL::SSL::Context::Client in _main.o
      _*Kemal::CLI::new<Array(String)>:Kemal::CLI in _main.o
  "_SSL_CTX_set_tmp_ecdh", referenced from:
      _*Kemal::CLI::new<Array(String)>:Kemal::CLI in _main.o
  "_SSL_set_tlsext_host_name", referenced from:
      _*OpenSSL::SSL::Socket::Client::new:context:sync_close:hostname<TCPSocket, OpenSSL::SSL::Context::Client, Bool, String>:OpenSSL::SSL::Socket::Client in _main.o
  "_lsquic_conn_get_ctx", referenced from:
      _*QUIC::Client::on_new_stream<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_conn_make_stream", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:208 in _main.o
  "_lsquic_conn_set_ctx", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:208 in _main.o
  "_lsquic_engine_check_settings", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:451 in _main.o
  "_lsquic_engine_connect", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:451 in _main.o
  "_lsquic_engine_destroy", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:208 in _main.o
  "_lsquic_engine_init_settings", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:451 in _main.o
  "_lsquic_engine_new", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:451 in _main.o
  "_lsquic_engine_packet_in", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:451 in _main.o
  "_lsquic_engine_process_conns", referenced from:
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:451 in _main.o
      _~procProc(Nil)@lib/lsquic/src/lsquic/client.cr:208 in _main.o
  "_lsquic_global_init", referenced from:
      ___crystal_main in _main.o
  "_lsquic_stream_conn", referenced from:
      _*QUIC::Client::on_new_stream<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_stream_flush", referenced from:
      _*QUIC::Client::on_write<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_stream_is_pushed", referenced from:
      _*QUIC::Client::on_new_stream<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_stream_readf", referenced from:
      _*QUIC::Client::on_read<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_stream_send_headers", referenced from:
      _*QUIC::Client::on_write<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_stream_shutdown", referenced from:
      _*QUIC::Client::on_write<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
      _*QUIC::Client::on_read<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_stream_wantread", referenced from:
      _*QUIC::Client::on_write<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
      _*QUIC::Client::on_read<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_stream_wantwrite", referenced from:
      _*QUIC::Client::on_new_stream<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
      _*QUIC::Client::on_write<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
  "_lsquic_stream_write", referenced from:
      _*QUIC::Client::on_write<Pointer(Void), Pointer(Void)>:Pointer(Void) in _main.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: execution of command failed with code: 1: `cc "${@}" -o /path/to/invidious/invidious  -rdynamic -L/Users/edwardloveall/.asdf/installs/crystal/0.35.1/embedded/lib /path/to/invidious/lib/lsquic/src/lsquic/ext/liblsquic.a -L/usr/local/Cellar/libyaml/0.2.5/lib -lyaml -lxml2 -lsqlite3 -lz `command -v pkg-config > /dev/null && pkg-config --libs --silence-errors libssl || printf %s '-lssl -lcrypto'` `command -v pkg-config > /dev/null && pkg-config --libs --silence-errors libcrypto || printf %s '-lcrypto'` -lpcre -lgc -lpthread /Users/edwardloveall/.asdf/installs/crystal/0.35.1/src/ext/libcrystal.a -L/usr/local/Cellar/libevent/2.1.12/lib -levent -liconv -ldl`

I'm sure I'm missing some library that I need to install or update but I have no idea what.

macOS Mojave (10.14.6)
Crystal 0.35.1

[alpha1125]

I can repeat that this is the same issue in
10.15.6
Crystal 0.35.1

[omarroth]

You'll want to build liblsquic.a for your platform according to the instructions here.

[edwardloveall]

Thank you @omarroth, for your help and this library.

I was able to build BoringSSL and LSQUIC. However the instructions you pointed to don't work from any directory that I tried. For example it says to run ar -x liblsquic.a but I'm not sure where liblsquic.a is supposed to be. It looks like all the .a files are in the same directory somewhere?

I'm sure I'm missing something obvious.

[omarroth]

liblsquic.a is built as lsquic/src/liblsquic/liblsquic.a
libssl.a and libcrypto.a are built as boringssl/ssl/libssl.a and boringssl/crypto/libcrypto.a, respectively.

All three were copied into the same directory for ease of use.

For building Invidious you'll want to replace the existing liblsquic.a in invidious/lib/lsquic/src/lsquic/ext/liblsquic.a with your version.

[edwardloveall]

That worked! The --strip-unneeded flag didn't exist on my strip command so I used -x -S instead. Here's the full instructions:

• Build BoringSSL
• Build LSQUIC
• make a new temp directory
• move (or copy) lsquic/src/liblsquic/liblsquic.a, boringssl/ssl/libssl.a, and boringssl/crypto/libcrypto.a to that temp directory
• Run the following from the temp directory:

$ ar -x liblsquic.a
$ ar -x libssl.a
$ ar -x libcrypto.a
$ rm *.a
$ ar rc liblsquic.a *.o
$ strip -x -S liblsquic.a 
$ ranlib liblsquic.a

(there are some things that look like errors on some commands but they didn't seem to affect the output)

• There's now a new liblsquic.a in your temp directory
• mv that to invidious/lib/lsquic/src/lsquic/ext/liblsquic.a
• Build Invidious with crystal build src/invidious.cr --release

[adnan360]

Thanks @omarroth for pointing me to this issue.

@edwardloveall
I had no problems building BoringSSL on my 32 bit Debian, but I had issues with LSQUIC. First cmake complained that BoringSSL headers not found. So I had to use cmake -DBORINGSSL_DIR=$BORINGSSL -DBORINGSSL_INCLUDE=$BORINGSSL/include . instead of the cmake command on README and it continued. But it wasn't the last one. On the make command at some point it showed this error:

...
[ 62%] Built target test_elision
Scanning dependencies of target test_min_heap
[ 62%] Building C object tests/CMakeFiles/test_min_heap.dir/test_min_heap.c.o
/home/invidious/lsquic/tests/test_min_heap.c: In function ‘test_min_heap’:
/home/invidious/lsquic/tests/test_min_heap.c:47:33: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
         lsquic_mh_insert(&heap, (void *) i, i);
                                 ^
/home/invidious/lsquic/tests/test_min_heap.c:62:33: error: cast to pointer from integer of different size [-Werror=int-to-pointer-cast]
         lsquic_mh_insert(&heap, (void *) i, i);
                                 ^
cc1: all warnings being treated as errors
make[2]: *** [tests/CMakeFiles/test_min_heap.dir/build.make:63: tests/CMakeFiles/test_min_heap.dir/test_min_heap.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1114: tests/CMakeFiles/test_min_heap.dir/all] Error 2
make: *** [Makefile:95: all] Error 2

After a lot of trial and error, I tried:

git reset && git checkout . && git clean -fdx
export CFLAGS+=" -Wno-int-to-pointer-cast -Wno-overflow -Wno-sign-compare"
cmake -DBORINGSSL_DIR=$BORINGSSL -DBORINGSSL_INCLUDE=$BORINGSSL/include .
make

And it worked. But I wonder if this is the right way to do it. Would suppressing all those errors cause any issues?

[github-actions]

This issue has been automatically marked as stale and will be closed in 30 days because it has not had recent activity and is much likely outdated. If you think this issue is still relevant and applicable, you just have to post a comment and it will be unmarked.

[unixfox]

New instructions for how to compile invidious without quic: https://docs.invidious.io/installation/#macos
Should fix the symbol issue.