Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustc deadlocks on FreeBSD #44433

Closed
jld opened this issue Sep 8, 2017 · 5 comments · Fixed by #46941
Closed

rustc deadlocks on FreeBSD #44433

jld opened this issue Sep 8, 2017 · 5 comments · Fixed by #46941
Labels
A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-bug Category: This is a bug. O-freebsd Operating system: FreeBSD

Comments

@jld
Copy link
Contributor

jld commented Sep 8, 2017

Steps to reproduce, on my FreeBSD/amd64 11.0 system: rustc --crate-type staticlib conftest.rs, where conftest.rs is:

pub extern fn hello() { println!("Hello world"); }

It's not 100%, but it usually happens after a few tries at most. I'll skip all the debugging and source-diving and just say that it's the same basic problem as #36905: threads A and B are both in LLVMRustWriteOutputFile, thread A gets to llvm::X86InstrFMA3Info::initGroupsOnce first and call_onces a lambda, then thread B enters llvm::X86InstrFMA3Info::initGroupsOnce and blocks in pthread_once with the std::call_once global mutex held, then thread A calls std::call_once reentrantly on a different once_flag and deadlocks.

But there might be a nicer solution than for #36905, because downstream has this patch:

--- src/librustc_llvm/build.rs.orig     2017-06-06 00:42:59 UTC
+++ src/librustc_llvm/build.rs
@@ -241,6 +241,8 @@ fn main() {
     let stdcppname = if target.contains("openbsd") {
         // OpenBSD has a particular C++ runtime library name
         "estdc++"
+    } else if target.contains("freebsd") {
+        "c++"
     } else if target.contains("netbsd") && llvm_static_stdcpp.is_some() {
         // NetBSD uses a separate library when relocation is required
         "stdc++_pic"

FreeBSD switched to libc++ in 10.0, on all platforms where they use Clang (apparently x86, little-endian ARM, and PPC), and 10.x is now the oldest supported release branch. libc++ appears to have a self-contained call_once that doesn't use pthread_once, and emprically the Rust 1.18.0 package available via pkg install doesn't deadlock.

So, it might be enough to upstream that patch and update the build environment to 10.x if it isn't already.

@mattico
Copy link
Contributor

mattico commented Sep 8, 2017

Looks like we're using 10.2 on ci:

URL=ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/10.2-RELEASE/base.txz

@jld
Copy link
Contributor Author

jld commented Sep 16, 2017

Thanks for the pointer. I took a (much) closer look at this, and I've gotten the build more or less working using Ubuntu's regular Clang package, which can cross-compile out of the box. I'll send a PR once I've gotten things cleaned up and tested a bit more.

@TimNN TimNN added A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-bug Category: This is a bug. O-freebsd Operating system: FreeBSD labels Sep 17, 2017
@jld
Copy link
Contributor Author

jld commented Sep 28, 2017

So I'm kind of stuck because, when I try to test locally, I run into #43982 unless I remove --enable-extended, which isn't maximally useful because I don't have Cargo and therefore can't build Cargo. This happens even without my changes, but the same container running on Travis CI seems to work, and I'm at a loss as to what differences there could be that would affect (apparently?) the type checker. I suppose I could just test what I can and send a PR and hope for the best, but if possible I'd prefer to understand what my local setup is doing wrong.

@valpackett
Copy link
Contributor

valpackett commented Oct 2, 2017

Noticed this on 12-CURRENT, compilers from rustup always lock up when compiling libloading, but it usually succeeds after a couple tries. It's nearly always libloading!

@valpackett
Copy link
Contributor

Hmm, the rust from ports (that doesn't have this problem IIRC??) is built with bundled llvm by default…

jld added a commit to jld/rust that referenced this issue Oct 7, 2017
…ng#44433.

The main goal here is to use FreeBSD's normal libc++, instead of
statically linking the libstdc++ packaged with GCC, because that
libstdc++ has bugs that cause rustc to deadlock inside LLVM.

But the easiest way to use libc++ is to switch the build from GCC to
Clang, and the Clang package in the Ubuntu image already knows how to
cross-compile (given a sysroot and preferably cross-binutils), so the
toolchain script now uses that instead of building a custom compiler.

This also de-duplicates the `build-toolchain.sh` script.
kennytm added a commit to kennytm/rust that referenced this issue Oct 11, 2017
…chton

Re-do the FreeBSD cross-builds to use Clang and libc++.  Fixes rust-lang#44433.

The main goal here is to use FreeBSD's normal libc++, instead of
statically linking the libstdc++ packaged with GCC, because that
libstdc++ has bugs that cause rustc to deadlock inside LLVM.

But the easiest way to use libc++ is to switch the build from GCC to
Clang, and the Clang package in the Ubuntu image already knows how to
cross-compile (given a sysroot and preferably cross-binutils), so the
toolchain script now uses that instead of building a custom compiler.

This also de-duplicates the `build-toolchain.sh` script.
bors added a commit that referenced this issue Dec 26, 2017
Re-do the FreeBSD cross-builds to use Clang and libc++. Fixes #44433

Reviving #45077, from @jld:

> The main goal here is to use FreeBSD's normal libc++, instead of
> statically linking the libstdc++ packaged with GCC, because that
> libstdc++ has bugs that cause rustc to deadlock inside LLVM.
>
> But the easiest way to use libc++ is to switch the build from GCC to
> Clang, and the Clang package in the Ubuntu image already knows how to
> cross-compile (given a sysroot and preferably cross-binutils), so the
> toolchain script now uses that instead of building a custom compiler.
>
> This also de-duplicates the build-toolchain.sh script.

#45077 was close but didn't quite make it.  I rebased @jld's work off the current `master` and started with that.

I was able to determine that this Travis error (#45077 (comment)) was ultimately caused by `src/librustc_llvm/build.rs` attempting to follow a wrong value in `LLVM_STATIC_STDCPP` (#45077 (comment)).

I looked at the downstream port for FreeBSD (https://svnweb.freebsd.org/ports/head/lang/rust/) and it seems like they do not use `--enable-llvm-static-stdcpp`.

Since `libc++` is included in the FreeBSD 10+ base system, we don't need to statically link it either?

So in b989428 I have set the FreeBSD build to not actually use `LLVM_STATIC_STDCPP`.

I was able to run `./src/ci/docker/run.sh` with both `dist-i686-freebsd` and `dist-x86_64-freebsd` successfully and in about 1 minute of testing it seemed like the dist-x86_64-freebsd results worked on a FreeBSD 11 system.

It should fix #44433, which seems to be affecting many potential users.  Also FreeBSD users should be able to `./x.py build` which should help anyone who wants to upstream fixes for FreeBSD.

Questions:

Does this approach seem to be the right way to go? Do we actually really want to statically link `libc++`? (I tried that here, but it ultimately ran into a roadblock on x86_64: #45077 (comment))

Can we rewrite the comment here to be more clear about why some systems aren't going to actually use this option:
https://github.com/rust-lang/rust/blob/b989428f7dec7b52d68bed6a21e9b5b0a8086267/src/bootstrap/compile.rs#L550-L553

How does this affect users of older FreeBSD systems? It seemed like no one was complaining about using a 10.3 base version in the thread for #45077.  FreeBSD seems to only officially support 10.3, 10.4, and 11.x right now, do we have to consider older users? The `libc++` stuff came in for FreeBSD 10, older FreeBSD used `libstdc++`.

Looks like @alexcrichton was leading the discussion on the previous issue:

r? @alexcrichton

Let me know what I can do to help get this through.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-bug Category: This is a bug. O-freebsd Operating system: FreeBSD
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants