can't find crate for core for x86_64-unknown-linux-musl target

[my4ng]

Checklist

• I've looked through the issues and pull requests for similar reports

Describe your issue

cross build --target x86_64-unknown-linux-musl -vv fails to compile while cargo build --target x86_64-unknown-linux-musl:

$ cross build --target x86_64-unknown-linux-musl -vv 
+ cargo metadata --format-version 1 --filter-platform x86_64-unknown-linux-musl
+ rustc --print sysroot
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/bin/podman
+ /usr/bin/podman run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=my4ng' --rm -v /home/my4ng/.xargo:/xargo:z -v /home/my4ng/.cargo:/cargo:z -v /cargo/bin -v /home/my4ng/Programming/tack:/project:z -v /home/my4ng/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -v /home/my4ng/Programming/tack/target:/target:z -w /project -i -t ghcr.io/cross-rs/x86_64-unknown-linux-musl:0.2.5 sh -c 'PATH=$PATH:/rust/bin cargo build --target x86_64-unknown-linux-musl -vv'
   Compiling libc v0.2.155
     Running `CARGO=/rust/bin/cargo CARGO_CRATE_NAME=libc CARGO_MANIFEST_DIR=/cargo/registry/src/index.crates.io-6f17d22bba15001f/libc-0.2.155 CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_DESCRIPTION='Raw FFI bindings to platform libraries like libc.
' CARGO_PKG_HOMEPAGE='https://github.com/rust-lang/libc' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libc CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/rust-lang/libc' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.2.155 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=155 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH=/target/debug/deps OUT_DIR=/target/x86_64-unknown-linux-musl/debug/build/libc-156c10c871e2c735/out rustc --crate-name libc --edition=2015 /cargo/registry/src/index.crates.io-6f17d22bba15001f/libc-0.2.155/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=107 --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=4b7a94ad2aac524a -C extra-filename=-4b7a94ad2aac524a --out-dir /target/x86_64-unknown-linux-musl/debug/deps --target x86_64-unknown-linux-musl -C linker=x86_64-linux-musl-gcc -L dependency=/target/x86_64-unknown-linux-musl/debug/deps -L dependency=/target/debug/deps --cap-lints warn --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_int128 --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor --cfg libc_non_exhaustive --cfg libc_long_array --cfg libc_ptr_addr_of --cfg libc_underscore_const_names --cfg libc_const_extern_fn`
error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-unknown-linux-musl` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-musl`

For more information about this error, try `rustc --explain E0463`.
error: could not compile `libc` (lib) due to 1 previous error

Caused by:
  process didn't exit successfully: `CARGO=/rust/bin/cargo CARGO_CRATE_NAME=libc CARGO_MANIFEST_DIR=/cargo/registry/src/index.crates.io-6f17d22bba15001f/libc-0.2.155 CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_DESCRIPTION='Raw FFI bindings to platform libraries like libc.
  ' CARGO_PKG_HOMEPAGE='https://github.com/rust-lang/libc' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libc CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/rust-lang/libc' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.2.155 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=155 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH=/target/debug/deps OUT_DIR=/target/x86_64-unknown-linux-musl/debug/build/libc-156c10c871e2c735/out rustc --crate-name libc --edition=2015 /cargo/registry/src/index.crates.io-6f17d22bba15001f/libc-0.2.155/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=107 --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=4b7a94ad2aac524a -C extra-filename=-4b7a94ad2aac524a --out-dir /target/x86_64-unknown-linux-musl/debug/deps --target x86_64-unknown-linux-musl -C linker=x86_64-linux-musl-gcc -L dependency=/target/x86_64-unknown-linux-musl/debug/deps -L dependency=/target/debug/deps --cap-lints warn --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_int128 --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor --cfg libc_non_exhaustive --cfg libc_long_array --cfg libc_ptr_addr_of --cfg libc_underscore_const_names --cfg libc_const_extern_fn` (exit status: 1)
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

I have done cargo clean but it does not help. It works for cargo:

$ cargo build --target x86_64-unknown-linux-musl                   
   Compiling libc v0.2.155
   Compiling memmap2 v0.9.4
   Compiling tac-k v0.2.1 (/home/my4ng/Programming/tack)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.51s

rustup show:

Default host: x86_64-unknown-linux-gnu
rustup home:  /home/my4ng/.rustup

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu (default)
nightly-x86_64-unknown-linux-gnu
1.74-x86_64-unknown-linux-gnu
1.75-x86_64-unknown-linux-gnu

installed targets for active toolchain
--------------------------------------

x86_64-unknown-linux-gnu
x86_64-unknown-linux-musl

active toolchain
----------------

stable-x86_64-unknown-linux-gnu (default)
rustc 1.78.0 (9b00956e5 2024-04-29)

It works on GitHub Runner ubuntu-latest using the taiki-e/install-action@cross action (see build.yml for details).

EDIT: tested with edge image, still the same error.

What target(s) are you cross-compiling for?

x86_64-unknown-linux-musl

Which operating system is the host (e.g computer cross is on) running?

• macOS
• Windows
• Linux / BSD
• other OS (specify in description)

What architecture is the host?

• x86_64 / AMD64
• arm32
• arm64 (including Mac M1)

What container engine is cross using?

• docker
• podman
• other container engine (specify in description)

cross version

cross 0.2.5

Example

No response

Additional information / notes

No response

[Emilgardis]

this smells like a filesystem problem with podman, possibly permission related.

does

podman run -it ubuntu:20.04 -v /home/my4ng/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro bash

ls /rust/lib/rustlib

show x86_64-unknown-linux-musl ?

[my4ng]

I ran podman run -v /home/my4ng/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -it ubuntu:22.04 bash:

$ ls /rust/lib/rustlib
ls: cannot access '/rust/lib/rustlib/components': Permission denied
components                                               manifest-rust-std-x86_64-unknown-linux-musl
etc                                                      manifest-rustc-x86_64-unknown-linux-gnu
manifest-cargo-x86_64-unknown-linux-gnu                  manifest-rustfmt-preview-x86_64-unknown-linux-gnu
manifest-clippy-preview-x86_64-unknown-linux-gnu         multirust-channel-manifest.toml
manifest-llvm-tools-preview-x86_64-unknown-linux-gnu     multirust-config.toml
manifest-rust-analyzer-preview-x86_64-unknown-linux-gnu  rust-installer-version
manifest-rust-docs-x86_64-unknown-linux-gnu              src
manifest-rust-src                                        x86_64-unknown-linux-gnu
manifest-rust-std-x86_64-unknown-linux-gnu               x86_64-unknown-linux-musl

Strangely components has the same permission as others in host:

.rw-r--r--@  347 my4ng 26 May 04:10 components
drwxr-xr-x@    - my4ng  3 May 01:49 etc
.rw-r--r--@ 1.5k my4ng  3 May 01:49 manifest-cargo-x86_64-unknown-linux-gnu
.rw-r--r--@  148 my4ng  3 May 01:49 manifest-clippy-preview-x86_64-unknown-linux-gnu
.rw-r--r--@  938 my4ng  3 May 01:49 manifest-llvm-tools-preview-x86_64-unknown-linux-gnu
.rw-r--r--@  147 my4ng  3 May 01:49 manifest-rust-analyzer-preview-x86_64-unknown-linux-gnu
.rw-r--r--@   24 my4ng  3 May 01:49 manifest-rust-docs-x86_64-unknown-linux-gnu
.rw-r--r--@ 109k my4ng  3 May 01:49 manifest-rust-src
.rw-r--r--@ 2.7k my4ng  3 May 01:49 manifest-rust-std-x86_64-unknown-linux-gnu
.rw-r--r--@ 3.4k my4ng 26 May 04:10 manifest-rust-std-x86_64-unknown-linux-musl
.rw-r--r--@ 1.1k my4ng  3 May 01:49 manifest-rustc-x86_64-unknown-linux-gnu
.rw-r--r--@  142 my4ng  3 May 01:49 manifest-rustfmt-preview-x86_64-unknown-linux-gnu
.rw-r--r--@ 745k my4ng 26 May 04:10 multirust-channel-manifest.toml
.rw-r--r--@  720 my4ng 26 May 04:10 multirust-config.toml
.rw-r--r--@    1 my4ng 26 May 04:10 rust-installer-version
drwxr-xr-x@    - my4ng  3 May 01:49 src
drwxr-xr-x@    - my4ng  3 May 01:49 x86_64-unknown-linux-gnu
drwxr-xr-x@    - my4ng 26 May 04:10 x86_64-unknown-linux-musl

But not in the container:

ls: cannot access 'components': Permission denied
total 884
drwxr-xr-x. 1 root root   1132 May 25 18:10 ./
drwxr-xr-x. 1 root root    268 May  2 15:49 ../
-?????????? ? ?    ?         ?            ? components
drwxr-xr-x. 1 root root    236 May  2 15:49 etc/
-rw-r--r--. 1 root root   1482 May  2 15:49 manifest-cargo-x86_64-unknown-linux-gnu
-rw-r--r--. 1 root root    148 May  2 15:49 manifest-clippy-preview-x86_64-unknown-linux-gnu
-rw-r--r--. 1 root root    938 May  2 15:49 manifest-llvm-tools-preview-x86_64-unknown-linux-gnu
-rw-r--r--. 1 root root    147 May  2 15:49 manifest-rust-analyzer-preview-x86_64-unknown-linux-gnu
-rw-r--r--. 1 root root     24 May  2 15:49 manifest-rust-docs-x86_64-unknown-linux-gnu
-rw-r--r--. 1 root root 108723 May  2 15:49 manifest-rust-src
-rw-r--r--. 1 root root   2728 May  2 15:49 manifest-rust-std-x86_64-unknown-linux-gnu
-rw-r--r--. 1 root root   3390 May 25 18:10 manifest-rust-std-x86_64-unknown-linux-musl
-rw-r--r--. 1 root root   1056 May  2 15:49 manifest-rustc-x86_64-unknown-linux-gnu
-rw-r--r--. 1 root root    142 May  2 15:49 manifest-rustfmt-preview-x86_64-unknown-linux-gnu
-rw-r--r--. 1 root root 745481 May 25 18:10 multirust-channel-manifest.toml
-rw-r--r--. 1 root root    720 May 25 18:10 multirust-config.toml
-rw-r--r--. 1 root root      1 May 25 18:10 rust-installer-version
drwxr-xr-x. 1 root root      8 May  2 15:49 src/
drwxr-xr-x. 1 root root     12 May  2 15:49 x86_64-unknown-linux-gnu/
drwxr-xr-x. 1 root root      6 May 25 18:10 x86_64-unknown-linux-musl/

[Emilgardis]

strange! maybe try recreating the file. does it work with cross +nightly?

[my4ng]

I have fixed it by uninstalling and reinstalling the stable toolchain, very strange indeed! Thanks for the help! I will close it now

[my4ng]

I have to reopen the issue since the problem creeps back up when I try to install a second target i686-unknown-linux-gnu. Whichever second target I try to add, it fails with unable to find core:

$ rustup toolchain uninstall stable
$ cross build --target i686-unknown-linux-gnu --release --locked -vv <- success
$ cross build --target x86_64-unknown-linux-musl --release --locked -vv <- failure

OR

$ rustup toolchain uninstall stable
$ cross build --target x86_64-unknown-linux-musl --release --locked -vv <- success
$ cross build --target i686-unknown-linux-gnu --release --locked -vv <- failure

[my4ng]

I've finally identified the cause: selinux is preventing access to components and the various .lib files from within the container.

According to the podman documentation:

To change a label in the container context, add either of two suffixes :z or :Z to the volume mount. These suffixes tell Podman to relabel file objects on the shared volumes. The z option tells Podman that two or more containers share the volume content. As a result, Podman labels the content with a shared content label. Shared volume labels allow all containers to read/write content. The Z option tells Podman to label the content with a private unshared label Only the current container can use a private volume. Relabeling walks the file system under the volume and changes the label on each file, if the volume has thousands of inodes, this process takes a long time, delaying the start of the container. If the volume was previously relabeled with the z option, Podman is optimized to not relabel a second time. If files are moved into the volume, then the labels can be manually change with the chcon -Rt container_file_t PATH command.

This makes sense why the first installed target works but not the second, since the first is installed before being relabelled with the z option, and the second one is done after.

Hence, I recommend adding an entry to the FAQ to both confirm this is indeed a permission issue (e.g. using ls -lZ) and suggest a solution chcon -Rt container_file_t ~/.rustup/toolchains.

[Emilgardis]

Maybe we could run chcon ourselves after running rustup target add

[my4ng]

That’s certainly the best, after detection that selinux is indeed installed on the system. It really only need to chcon the specific lib folder, but there doesnt appear to be any harm in chcon the toolchains folder entirely.

[Emilgardis]

Yes, I think the way to mitigate is

1. relabel the toolchain folder if we've modified it ourself, possibly only touching what we know can change
2. emit a hint/suggestion if build fails and selinux is enabled here

and ofc we should document this caveat

[my4ng]

Unfortunately that won't be enough. If the user installed the rust-std for the specific target after the initial run of cross themselves, then it won't get relabelled. I think the only way is on a per-use basis, i.e. if we are going to use a specific target, we install (if does not exist) and then check, and if necessarily, change permission.

and ofc we should document this caveat

Agreed. I can submit a PR for both the code and docs if you want.

[spemmons]

any movement on this? was there a PR submitted that needs to be merged? just checking, thanks!

[my4ng]

any movement on this? was there a PR submitted that needs to be merged? just checking, thanks!

Sorry have been busy working on other things but it should be a relatively easy fix. It appears chcon -Rt container_file_t ~/.rustup/toolchains needs to be run every time, in case rustup update has overwritten with a new release (esp. for nightly).

@spemmons unfortunately I don't have a Linux machine with me at the moment and can maybe have a PR out next weekend at the earliest. If you want to work on it yourself, that's all good with me.

[spemmons]

curiously, i have the same problem w/ docker, not podman -- i can look into it but i'm not 100% clear on what needs to be done

[my4ng]

@spemmons I believe it is the same issue as I explained in a previous comment, essentially when the toolchain updates, the folder no longer has the correct SELinux type, and therefore cannot be used by podman/docker. :z/Z is lazy and will only change it once at the first time.