Segfault in alpine docker image while initializing client

[Hakukano]

This issue can only be reproduced in Alpine docker image: alpine or Rust alpine image: rust:alpine. Debian based images work just fine with the same code.

Min code

Running in actix-web::main

use reqwest::Client;
...
let client = Client::new(); // segfault (docker exist code 139) here
// Or using builder
let client = Client::builder().build(); // segfault (docker exist code 139) here
...

Error log

There isn't any log whatsoever. According to docker exit code 139, there is a segfault occurred.

Possible cause

I don't think it's caused by missing libraries or the log should show what's missing but instead, there are no logs.
Maybe something wrong with alpine's musl libc vs glibc?

[seanmonstar]

Well that's pretty strange. Could you try debugging a little more, perhaps using strace to see what calls the program gets to before crashing?

[Hakukano]

Here is the strace:

Before crash

...
rt_sigprocmask(SIG_BLOCK, [QUIT], [], 8) = 0
brk(0x55555714e000)                     = 0x55555714e000
rt_sigaction(SIGQUIT, {sa_handler=0x7f64849224e0, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO|SA_NOCLDSTOP, sa_restorer=0x7f648496510f}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
fcntl(46, F_DUPFD_CLOEXEC, 0)           = 51
fcntl(51, F_SETFD, FD_CLOEXEC)          = 0
fcntl(51, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
epoll_ctl(3, EPOLL_CTL_ADD, 51, {EPOLLIN|EPOLLPRI|EPOLLOUT|EPOLLET, {u32=3, u64=3}}) = 0
epoll_pwait(3, [{EPOLLOUT, {u32=0, u64=0}}, {EPOLLOUT, {u32=1, u64=1}}, {EPOLLOUT, {u32=2, u64=2}}, {EPOLLOUT, {u32=3, u64=3}}], 1024, -1, NULL, 8) = 4

Crash

epoll_pwait(3,  <unfinished ...>)       = ?
+++ killed by SIGSEGV (core dumped) +++

[Hakukano]

Additionally, I have Valgrind log here:

==1== Thread 2 actix-rt:worker::
==1== Jump to the invalid address stated on the next line
==1==    at 0x0: ???
==1==    by 0x31EC3D: native_tls::imp::TlsConnector::new (in /fwcheck)
==1==    by 0x31F2B4: native_tls::TlsConnectorBuilder::build (in /fwcheck)
==1==    by 0x31C423: reqwest::connect::Connector::new_default_tls (in /fwcheck)
==1==    by 0x305706: reqwest::async_impl::client::ClientBuilder::build (in /fwcheck)
==1==    by 0x20A7F0: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1B06A1: <actix_web::handler::HandlerServiceResponse<T,R> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BA4A8: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BACC9: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1D6556: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1EBAA8: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x357780: <core::pin::Pin<P> as core::future::future::Future>::poll (in /fwcheck)
==1==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==1==
==1==
==1== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==1==  Bad permissions for mapped region at address 0x0
==1==    at 0x0: ???
==1==    by 0x31EC3D: native_tls::imp::TlsConnector::new (in /fwcheck)
==1==    by 0x31F2B4: native_tls::TlsConnectorBuilder::build (in /fwcheck)
==1==    by 0x31C423: reqwest::connect::Connector::new_default_tls (in /fwcheck)
==1==    by 0x305706: reqwest::async_impl::client::ClientBuilder::build (in /fwcheck)
==1==    by 0x20A7F0: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1B06A1: <actix_web::handler::HandlerServiceResponse<T,R> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BA4A8: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BACC9: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1D6556: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1EBAA8: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x357780: <core::pin::Pin<P> as core::future::future::Future>::poll (in /fwcheck)
==1==
==1== HEAP SUMMARY:
==1==     in use at exit: 0 bytes in 0 blocks
==1==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==1==
==1== All heap blocks were freed -- no leaks are possible
==1==
==1== Use --track-origins=yes to see where uninitialised values come from
==1== For lists of detected and suppressed errors, rerun with: -s
==1== ERROR SUMMARY: 5 errors from 2 contexts (suppressed: 0 from 0)

valgrind: the 'impossible' happened:
   main(): signal was supposed to be fatal

host stacktrace:
==1==    at 0x58031AD2: show_sched_status_wrk (m_libcassert.c:388)
==1==    by 0x58031D26: report_and_quit (m_libcassert.c:459)
==1==    by 0x58031F3D: panic (m_libcassert.c:535)
==1==    by 0x58031F3D: vgPlain_core_panic_at (m_libcassert.c:540)
==1==    by 0x58031F45: vgPlain_core_panic (m_libcassert.c:545)
==1==    by 0x58081326: shutdown_actions_NORETURN (m_main.c:2268)
==1==    by 0x580AA3C7: run_a_thread_NORETURN (syswrap-linux.c:203)
==1==    by 0x580AA465: vgModuleLocal_start_thread_NORETURN (syswrap-linux.c:320)
==1==    by 0x5807FA72: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1==    by 0xDEADBEEFDEADBEEE: ???
==1==    by 0xDEADBEEFDEADBEEE: ???
==1==    by 0xDEADBEEFDEADBEEE: ???

sched status:
  running_tid=2


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

[seanmonstar]

Ah interesting. It could be the crash is from native-tls... You could file an issue there. Also, you could try disabling native-tls in reqwest and enabling rustls.

[onelson]

In the past there have been caveats cross-compiling with musl around getting openssl to link correctly. Workarounds involved explicitly adding an extern crate openssl; to all the binary targets in your project and somehow the ordering was significant... emk/rust-musl-builder#69 (comment)

Using rustls should sidestep the issue since you'd no longer need to link openssl.

[Hakukano]

Enabling rustls works for me! Thanks guys!

[Daktic]

Can you expand on what you mean by enable rustls to sidestep the tls issue?

I have been stuck on this for days, and can't get my head wrapped around how to fix this.

I have a really simple implementation here where I just create an actix-web route, initialize a new reqwest client, and I segfault when calling the route. I can see that it is related to the TLS failing to do something, but can't seem to get anything to work to fix it.

https://github.com/Daktic/mock_reqwest