Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

futures-macro (pulled in by futures-util) appears not to compile on musl #2242

Closed
jbaublitz opened this issue Oct 25, 2020 · 30 comments
Closed

Comments

@jbaublitz
Copy link

I have a CI pipeline that tests my project on glibc and musl and it appears that when I pull in futures-util as a dependency, this compiles just fine on glibc but does not compile on musl.

Here is the build log: https://travis-ci.org/github/jbaublitz/neli/jobs/738716295

This appears to be due to futures-macro exporting all of its procedural macros with #[proc_macro_hack] which is not allowed on musl.

I know that rust and musl have had problems in the past related to proc macros and I'm wondering if this is the right place to report this or if this should go into the rust repo.

@taiki-e
Copy link
Member

taiki-e commented Oct 25, 2020

I believe this issue was fixed in Rust 1.44+ (see rust-lang/rust#40174). Can you check musl-dev is installed?

@jbaublitz
Copy link
Author

This appears to be a different issue as I do have musl-dev installed (see here) and it is able to find the proc-macro crate. It is simply complaining about the usage of #[proc_macro_hack] which is not what the linked issue seems to be about. I read that one over before filing this one.

@taiki-e
Copy link
Member

taiki-e commented Oct 25, 2020

Hmm, I'm not sure what caused this problem, but this should eventually be fixed by dtolnay/proc-macro-hack#64.

It should be able to compile by disabling futures-util's default features until that pr is merged or another workaround is found.

futures-util = { version = "0.3", default-features = false, features = ["std"] }

@jbaublitz
Copy link
Author

Oh thanks for putting up a PR! Do you want me to test anything for now?

@loewenheim
Copy link

I'm having the same or at least a similar issue. Adding

futures-util = { version = "0.3", default-features = false, features = ["std"] }

to Cargo.toml doesn't fix it. Here's my output:

❯ cargo build
   Compiling futures-macro v0.3.7
   Compiling pin-project-internal v0.4.27
   Compiling serde_derive v1.0.117
   Compiling thiserror-impl v1.0.21
error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:31:1
   |
31 | pub fn join_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:37:1
   |
37 | pub fn try_join_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:43:1
   |
43 | pub fn select_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:49:1
   |
49 | pub fn select_biased_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libproc_macro_hack-59bcb376d337da05.so)
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:24:5
   |
24 | use proc_macro_hack::proc_macro_hack;
   |     ^^^^^^^^^^^^^^^

error: aborting due to 5 previous errors

error: could not compile `futures-macro`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

I don't know what the deal is with that specific version of glibc being required. The version I have available is 2.31.

@taiki-e
Copy link
Member

taiki-e commented Oct 28, 2020

@loewenheim I believe you have another dependency that enables async-await feature of futures-util. Can you show me the output of cargo tree -i -p futures-util?

❯ cargo build
Compiling futures-macro v0.3.7

@loewenheim
Copy link

Here's the output:

❯ cargo tree -i -p futures-util
futures-util v0.3.7
├── futures v0.3.7
│   └── teloxide v0.3.2
│       └── latexbot-rs v0.1.0 (/home/sebastian/git/latexbot-rs)
├── futures-executor v0.3.7
│   └── futures v0.3.7 (*)
├── h2 v0.2.7
│   └── hyper v0.13.8
│       ├── hyper-tls v0.4.3
│       │   └── reqwest v0.10.8
│       │       └── teloxide v0.3.2 (*)
│       └── reqwest v0.10.8 (*)
├── hyper v0.13.8 (*)
├── latexbot-rs v0.1.0 (/home/sebastian/git/latexbot-rs)
└── reqwest v0.10.8 (*)

@taiki-e
Copy link
Member

taiki-e commented Oct 28, 2020

@loewenheim

Here's the output:

Seems async-await feature is enabled by teloxide.

@taiki-e
Copy link
Member

taiki-e commented Oct 28, 2020

Since I merged #2243, this should be fixed on the master branch.

@loewenheim
Copy link

Unfortunately, that hasn't seemed to fix it for me. I now get

❯ cargo tree -i -p futures-util
futures-util v0.3.7 (https://github.com/rust-lang/futures-rs?rev=4122f3d5#4122f3d5)
├── futures v0.3.7
│   └── teloxide v0.3.2
│       └── latexbot-rs v0.1.0 (/home/sebastian/git/latexbot-rs)
├── futures-executor v0.3.7
│   └── futures v0.3.7 (*)
├── h2 v0.2.7
│   └── hyper v0.13.8
│       ├── hyper-tls v0.4.3
│       │   └── reqwest v0.10.8
│       │       └── teloxide v0.3.2 (*)
│       └── reqwest v0.10.8 (*)
├── hyper v0.13.8 (*)
└── reqwest v0.10.8 (*)

but the build still fails with exactly the same error as before.

@taiki-e
Copy link
Member

taiki-e commented Oct 28, 2020

@loewenheim Replacing dependencies with git doesn't replace dependencies of dependencies with git. It needs to add a patch section like the following to workspace root's Cargo.toml.

[patch.crates-io]
futures = { git = "https://github.com/rust-lang/futures-rs" }
futures-core = { git = "https://github.com/rust-lang/futures-rs" }
futures-channel = { git = "https://github.com/rust-lang/futures-rs" }
futures-executor = { git = "https://github.com/rust-lang/futures-rs" }
futures-io = { git = "https://github.com/rust-lang/futures-rs" }
futures-macro = { git = "https://github.com/rust-lang/futures-rs" }
futures-sink = { git = "https://github.com/rust-lang/futures-rs" }
futures-task = { git = "https://github.com/rust-lang/futures-rs" }
futures-util = { git = "https://github.com/rust-lang/futures-rs" }

(In any case, I should create a new release...)

@loewenheim
Copy link

Still no luck, I'm afraid. Though I'm not entirely sure that the problems are caused by this issue.

❯ cargo build
   Compiling futures-macro v0.3.7 (https://github.com/rust-lang/futures-rs#4122f3d5)
   Compiling pin-project v1.0.1
   Compiling tokio v0.2.22
   Compiling serde v1.0.117
error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libpin_project_internal-71a017b8e79917db.so)
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/pin-project-1.0.1/src/lib.rs:85:9
   |
85 | pub use pin_project_internal::pin_project;
   |         ^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: could not compile `pin-project`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libserde_derive-428f432050fd432d.so)
   --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.117/src/lib.rs:286:1
    |
286 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:31:1
   |
31 | pub fn join_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:37:1
   |
37 | pub fn try_join_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:43:1
   |
43 | pub fn select_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:49:1
   |
49 | pub fn select_biased_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libproc_macro_hack-59bcb376d337da05.so)
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:24:5
   |
24 | use proc_macro_hack::proc_macro_hack;
   |     ^^^^^^^^^^^^^^^

error: aborting due to 5 previous errors

error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libtokio_macros-b907bffc258ed627.so)
   --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.22/src/lib.rs:403:13
    |
403 |     pub use tokio_macros::select_priv_declare_output_enum;
    |             ^^^^^^^^^^^^

error: aborting due to previous error

error: build failed

@taiki-e
Copy link
Member

taiki-e commented Oct 30, 2020

@loewenheim
Hmm, it seems all proc-macros failed to compile. Can you show me the version of rustc (rustc --version --verbose)?
Also, if you installed rustc using a way other than rustup, that may be the cause.
If updating the compiler and using the toolchain installed via rustup doesn't fix the problem, you may be able to fix it by using -C target-feature=-crt-static (rust-lang/rust#59302):

RUSTFLAGS="-C target-feature=-crt-static" cargo build

@loewenheim
Copy link

I installed rust via rustup. rustc --version --verbose outputs

rustc 1.47.0 (18bf6b4f0 2020-10-07)
binary: rustc
commit-hash: 18bf6b4f01a6feaf7259ba7cdae58031af1b7b39
commit-date: 2020-10-07
host: x86_64-unknown-linux-gnu
release: 1.47.0
LLVM version: 11.0

Adding the target-feature flag didn't fix the problem. I'm very thankful for your assistance, but I'm honestly not sure investigating this further is worth your time—I'm using NixOS and there's just been a major update, the problem may be related to that.

This was referenced Nov 3, 2020
@taiki-e
Copy link
Member

taiki-e commented Nov 6, 2020

Possibly related: rust-lang/rust#78210
I think it's a slightly different issue, but the same workaround (specify target explicitly) may work:

RUSTFLAGS='-C target-feature=+crt-static' cargo build --target x86_64-unknown-linux-gnu

@loewenheim
Copy link

I finally figured out what the issue on my end was—it was a NixOS problem, nothing to do with the library at all.

@taiki-e
Copy link
Member

taiki-e commented Nov 7, 2020

@loewenheim

it was a NixOS problem,

Is the problem tracked on the NixOS side? (I searched for it a few days ago but I couldn't find any related issues...)

@loewenheim
Copy link

I'm sorry, I was being unclear. It was a problem with my specific NixOS configuration.

@taiki-e
Copy link
Member

taiki-e commented Nov 8, 2020

Ah, ok, thanks for clarifying.

@taiki-e
Copy link
Member

taiki-e commented Nov 11, 2020

@loewenheim: It seems some people encountered the same problem as you encountered (e.g., #2253). If you could tell us information about what configuration was the cause is very helpful.

@taiki-e
Copy link
Member

taiki-e commented Nov 11, 2020

@jbaublitz: We have published 0.3.8. Can you see if it fixed the problem you encountered?

@Philipp-M
Copy link

This may help:

I've encountered the same issue (on NixOS) in an older rust nightly version (I believe it was the nightly from 22th October), after I've updated to the most recent (8th November) (using rustup) the issue was gone. @loewenheim and all NixOS users have you tried the most recent nightly rust toolchain?

I'm not sure though if the rust toolchain update was the only necessary fix. For me it stopped working (with the same issue as @loewenheim had), and after some time (I didn't actively pursue this issue) it started working again...

@taiki-e
Copy link
Member

taiki-e commented Nov 11, 2020

error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libproc_macro_hack-59bcb376d337da05.so)

This error is likely https://superuser.com/questions/537683/how-to-fix-lib-x86-64-linux-gnu-libc-so-6-version-glibc-2-14-not-found.

That means the program was compiled against glibc version 2.14, and it requires that version to run, but your system has an older version installed. You'll need to either recompile the program against the version of glibc that's on your system, or install a newer version of glibc (the "libc6" package in Debian).

@loewenheim
Copy link

In my case, it worked like this: my system depends on the 20.09 channel of nixpkgs.

  # flake.nix
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-20.09";

I also make local dev environments for Rust projects. I use nix + direnv for this. I put this in the file flake.nix in the project directory:

{
  description = "Rust dev environment";
  outputs = { self, nixpkgs }: {
    # setup the devShell for x86_64-linux.
    devShell.x86_64-linux = with nixpkgs.legacyPackages.x86_64-linux;
      mkShell {
        nativeBuildInputs = [ pkgconfig rust-bindgen ];
        buildInputs = [
          # dependencies go here
          alsaLib
          libudev
          openssl
          …
        ];
      };
  };
}

and this in .envrc:

# reload when these files change
watch_file flake.nix
watch_file flake.lock
# load the flake devShell
eval "$(nix print-dev-env)"

This automatically rebuilds the dev environment whenever flake.nix changes.

The flake.nix above implicitly depends on nixpkgs. I assumed that it would use the same version as my system configuration above, but this was actually not the case—it used the unstable version. This was the cause of the problem. Some things were compiled with glibc 2.32 from unstable nixpkgs and some with 2.31 from 20.09. There are two ways to fix this.

  1. Add an explicit line to the project's flake.nix that specifies the version of nixpkgs to use, like at the start of this post.
  2. Modify the flake registry; see this blog post. By default, it says global flake:nixpkgs github:NixOS/nixpkgs. You can modify this using e.g. nix registry add flake:nixpkgs github:NixOS/nixpkgs/nixos-20.09. Running nix registry list now shows user flake:nixpkgs github:NixOS/nixpkgs/nixos-20.09 instead of the previous line. This means that you have overridden the standard definition of the nixpkgs flake and nixpkgs will refer to the 20.09 branch unless specified otherwise.

After doing either 1. or 2., you also have to update the flake using nix flake update --update-input nixpkgs --commit-lock-file.

@taiki-e
Copy link
Member

taiki-e commented Nov 13, 2020

Thanks @loewenheim for the detailed information!
If I understand correctly, it can also be fixed by removing the old build artifacts that were generated before updating to nixos-20.09 (i.e., run cargo clean), right? EDIT: This guess seems not correct

@loewenheim
Copy link

cargo clean wasn't sufficient in my case at least.

@taiki-e
Copy link
Member

taiki-e commented Nov 13, 2020

@jbaublitz: I think this is the solution to the error you encountered. (tl; dr, It seems that it can be fixed by installing libc6-compat or gcompat.)

@jbaublitz
Copy link
Author

@taiki-e I'll try to test this out when I get the change in my CI pipeline.

@taiki-e taiki-e added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author label Nov 23, 2020
@jbaublitz
Copy link
Author

Hi @taiki-e, sorry this took so long! I was previously testing on a very handspun Ubuntu docker image. I've switched over to alpine with libc6-compat and it works perfectly. Thanks for your help! Closing.

@taiki-e taiki-e added C-question and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author labels Nov 25, 2020
@kristoff3r
Copy link

For future NixOS users googling this error message, this can also happen when using rustup and updating your system / glibc version. Clearing ~/.rustup and reinstalling the needed toolchains solved it for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants