Cargo panic in ResolvedFeatures::activated_features_int with unstable bindeps feature

[phlip9]

Problem

While experimenting with using the unstable bindeps feature, I stumbled upon a panic. Here's a repo with a minimized failing case: https://github.com/phlip9/cargo-bindep-panic. Simply clone it and run a cargo command like cargo build or cargo tree to see the panic.

In plain english, we have a crate, will-get-bindeped, that's targetting some other arch like wasm or x86_64-fortanix-unknown-sgx in our case. At the root is a crate, top-level-that-depends-on-bindep, that wants to depend on the built binary of will-get-bindeped (hence the bindeps feature).

This binary, will-get-bindeped, depends on dep-that-depends-on-build-only-dep that has a build-time dependency. This build-only-dep represents something like protoc-rust that needs to run on the host and compile some .proto's or w/e.

build-only-dep has a cfg(unix) (this seems important!) on some-leaf-dep. If some-leaf-dep is just an unconditional dependency or even a cfg(windows), the panic no longer triggers.

Visually, the crate graph looks like:

top-level-that-depends-on-bindep
  |
  | [dependencies] artifact = "bin", target = "x86_64-fortanix-unknown-sgx"
  |     - or -
  | [build-dependencies] artifact = "bin", target = "x86_64-fortanix-unknown-sgx"
  V
will-get-bindeped
  |
  | [dependencies]
  V
dep-that-depends-on-build-only-dep
  |
  | [build-dependencies]
  V
build-only-dep
  |
  | [target.'cfg(unix)'.dependencies]
  V
some-leaf-dep

Steps

$ git clone https://github.com/phlip9/cargo-bindep-panic
$ cd cargo-bindep-panic
$ RUST_BACKTRACE=1 cargo check

thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "some-leaf-dep", version: "0.1.0", source: "/Users/phlip9/dev/cargo-bindep-panic/some-leaf-dep" } ArtifactDep(CompileTarget { name: "x86_64-fortanix-unknown-sgx" })

Stack backtrace:
   0: std::backtrace::Backtrace::create
   1: <anyhow::Error>::msg::<alloc::string::String>
   2: <cargo::core::resolver::features::ResolvedFeatures>::activated_features_int
   3: cargo::core::compiler::unit_dependencies::new_unit_dep_with_profile
   4: cargo::core::compiler::unit_dependencies::compute_deps
   5: cargo::core::compiler::unit_dependencies::deps_of
   6: cargo::core::compiler::unit_dependencies::deps_of
   7: cargo::core::compiler::unit_dependencies::deps_of
   8: cargo::core::compiler::unit_dependencies::deps_of
   9: cargo::core::compiler::unit_dependencies::deps_of
  10: cargo::core::compiler::unit_dependencies::deps_of
  11: cargo::core::compiler::unit_dependencies::deps_of
  12: cargo::core::compiler::unit_dependencies::deps_of_roots
  13: cargo::core::compiler::unit_dependencies::build_unit_dependencies
  14: cargo::ops::cargo_compile::create_bcx
  15: cargo::ops::cargo_compile::compile_ws
  16: cargo::ops::cargo_compile::compile
  17: cargo::commands::check::exec
  18: cargo::cli::main
  19: cargo::main
  20: std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
  21: std::rt::lang_start::<()>::{closure#0}
  22: std::rt::lang_start_internal
  23: _main', src/cargo/core/resolver/features.rs:322:14

Other cargo commands also fail immediately:

$ RUST_BACKTRACE=1 cargo tree
thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "will-get-bindeped", version: "0.1.0", source: "/Users/phlip9/dev/cargo-bindep-panic/will-get-bindeped" } HostDep

Stack backtrace:
   0: std::backtrace::Backtrace::create
   1: <anyhow::Error>::msg::<alloc::string::String>
   2: <cargo::core::resolver::features::ResolvedFeatures>::activated_features_int
   3: cargo::ops::tree::graph::add_pkg
   4: cargo::ops::tree::graph::add_pkg
   5: cargo::ops::tree::graph::build
   6: cargo::ops::tree::build_and_print
   7: cargo::commands::tree::exec
   8: cargo::cli::main
   9: cargo::main
  10: std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
  11: std::rt::lang_start::<()>::{closure#0}
  12: std::rt::lang_start_internal
  13: _main', src/cargo/core/resolver/features.rs:322:14

Possible Solution(s)

No response

Notes

No response

Version

$ cargo version --verbose

cargo 1.73.0-nightly (45782b6b8 2023-07-05)
release: 1.73.0-nightly
commit-hash: 45782b6b8afd1da042d45c2daeec9c0744f72cc7
commit-date: 2023-07-05
host: aarch64-apple-darwin
libgit2: 1.6.4 (sys:0.17.2 vendored)
libcurl: 7.88.1 (sys:0.4.63+curl-8.1.2 system ssl:(SecureTransport) LibreSSL/3.3.6)
ssl: OpenSSL 1.1.1u  30 May 2023
os: Mac OS 13.4.1 [64-bit]

[weihanglo]

Thanks for the report! Does it share the same root cause with #10593? Looks a bit similar.

[phlip9]

Definitely looks similar... One weird thing I'm also noticing:

If we make some-leaf-dep unconditional

[package]
name = "build-only-dep"
version = "0.1.0"
edition = "2021"

# [target.'cfg(unix)'.dependencies] # << turn this off

[dependencies] # << make this a normal dependency
some-leaf-dep = { path = "../some-leaf-dep" }

Then something like cargo check will work (!) but cargo tree will still panic (?!):

$ cargo check
   Compiling some-leaf-dep v0.1.0
    Checking build-only-dep v0.1.0
   Compiling dep-that-depends-on-build-only-dep v0.1.0
    Checking will-get-bindeped v0.1.0
   Compiling top-level-that-depends-on-bindep v0.1.0
    Finished dev [unoptimized + debuginfo] target(s) in 0.48s

$ cargo tree

thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "will-get-bindeped", version: "0.1.0", source: "/Users/phlip9/dev/cargo-bindep-panic/will-get-bindeped" } HostDep', src/cargo/core/resolver/features.rs:322:14

[weihanglo]

The feature resolution path is slightly different between cargo tree and cargo check. That could be the reason. I'll keep this open for cargo check, and #10593 for cargo tree.

[rukai]

In case its helpful for anyone, I turned this excellent reproduction into two integration tests:

Expand...

The first test reproduces the cargo tree issue, its quite easy to hit.
The second test reproduces the cargo check issue, its quite specific to hit.

#[cargo_test]
fn resolver2_bindep_crosscompile() {
    if cross_compile::disabled() {
        return;
    }
    // At least on linux the i686-unknown-linux-gnu target triggered a panic.
    let target = cross_compile::alternate();

    let p = project()
        .file(
            "Cargo.toml",
            &r#"
                [package]
                name = "foo"
                version = "0.0.0"
                authors = []
                resolver = "2"

                [dependencies]
                bindep = { path = "bindep", artifact = "bin", target = "$TARGET" }
            "#
            .replace("$TARGET", target),
        )
        .file("src/lib.rs", "")
        .file("bindep/Cargo.toml", &basic_manifest("bindep", "0.0.0"))
        .file("bindep/src/main.rs", "fn main() {}")
        .build();

    p.cargo("check -Z bindeps")
        .masquerade_as_nightly_cargo(&["bindeps"])
        .with_stderr_contains(
            r#"[COMPILING] bindep v0.0.0 ([CWD]/bindep)
[CHECKING] foo v0.0.0 ([CWD])
[FINISHED] dev [unoptimized + debuginfo] target(s) in [..]"#,
        )
        .with_status(0)
        .run();

    p.cargo("tree -Z bindeps")
        .masquerade_as_nightly_cargo(&["bindeps"])
        .with_stdout(
            "\
foo v0.0.0 ([CWD])
└── bindep v0.0.0 ([CWD]/bindep)",
        )
        .with_status(0)
        .run();
}

#[cargo_test]
fn check_bindep_depending_on_build_only_dep_with_optional_dep() {
    if cross_compile::disabled() {
        return;
    }
    // i686-unknown-linux-gnu did not trigger a panic so we need to manually specify wasm32-unknown-unknown instead of relying on cross_compile::alternate()
    // This is possibly because the `cfg(unix)` later on needs to evaluate to false on the cross compile but true on the host.
    let target = "wasm32-unknown-unknown";

    let p = project()
        .file(
            "Cargo.toml",
            &r#"
                [package]
                name = "foo"
                version = "0.0.0"
                authors = []
                resolver = "2"

                [dependencies]
                bindep = { path = "bindep", artifact = "bin", target = "$TARGET" }
            "#
            .replace("$TARGET", target),
        )
        .file("src/lib.rs", "")
        .file(
            "bindep/Cargo.toml",
            &r#"
                [package]
                name = "bindep"
                version = "0.0.0"
                authors = []

                [dependencies]
                depends-on-build-only-dep = { path = "../depends-on-build-only-dep" }
            "#,
        )
        .file("bindep/src/main.rs", "fn main() {}")
        .file(
            "depends-on-build-only-dep/Cargo.toml",
            &r#"
                [package]
                name = "depends-on-build-only-dep"
                version = "0.0.0"
                authors = []

                [build-dependencies]
                build-only-dep = { path = "../build-only-dep" }
            "#,
        )
        .file("depends-on-build-only-dep/src/lib.rs", "")
        .file("depends-on-build-only-dep/build.rs", "fn main() {}")
        .file(
            "build-only-dep/Cargo.toml",
            &r#"
                [package]
                name = "build-only-dep"
                version = "0.0.0"
                authors = []

                [target.'cfg(unix)'.dependencies]
                some-leaf-dep = { path = "../some-leaf-dep" }
            "#,
        )
        .file("build-only-dep/src/lib.rs", "")
        .file(
            "some-leaf-dep/Cargo.toml",
            &basic_manifest("some-leaf-dep", "0.0.1"),
        )
        .file("some-leaf-dep/src/lib.rs", "")
        .build();
    p.cargo("check -Z bindeps")
        .masquerade_as_nightly_cargo(&["bindeps"])
        .with_stderr_contains(
            r#"[COMPILING] bindep v0.0.0 ([CWD]/bindep)
[CHECKING] foo v0.0.0 ([CWD])
[FINISHED] dev [unoptimized + debuginfo] target(s) in [..]"#,
        )
        .with_status(0)
        .run();
}

[Veykril]

Managed to trigger this in rust-analyzer by accident as well (without any special targets set afaik), so if it helps, just putting a reproducer here as well: https://github.com/Veykril/rust-analyzer/tree/cargo-panic

❯ cargo check -p proc-macro-test
thread 'main' panicked at src/cargo\core\resolver\features.rs:323:14:
activated_features for invalid package: features did not find PackageId { name: "proc-macro-test", version: "0.0.0", source: "C:\\workspace\\rust\\rust-analyzer\\crates\\proc-macro-
srv\\proc-macro-test" } NormalOrDev
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[pscott31]

I have bumped into this problem too, trying to compile a wasm32-unknown-unknown target (Leptos frontend) for embedding into an, e.g. aarch64-apple-darwin binary that serves it up and have had to fall back to calling cargo inside build.rs which is suboptimal for lots of reasons. I'd love to help fixing it but this is way over my head, if there is anything I can do to test or help please let me know:

➜  happenings-ng git:(main) ✗ cargo version         
cargo 1.76.0-nightly (9765a449d 2023-11-17)
➜  happenings-ng git:(main) ✗ RUST_BACKTRACE=1 cargo build -Z bindeps
thread 'main' panicked at src/cargo/core/resolver/features.rs:323:14:
activated_features for invalid package: features did not find PackageId { name: "libc", version: "0.2.151", source: "registry `crates-io`" } ArtifactDep(CompileTarget { name: "wasm32-unknown-unknown" })
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: cargo::core::compiler::unit_dependencies::new_unit_dep_with_profile
   4: cargo::core::compiler::unit_dependencies::compute_deps
   5: cargo::core::compiler::unit_dependencies::deps_of
   6: cargo::core::compiler::unit_dependencies::deps_of
   7: cargo::core::compiler::unit_dependencies::deps_of
   8: cargo::core::compiler::unit_dependencies::deps_of
   9: cargo::core::compiler::unit_dependencies::deps_of
  10: cargo::core::compiler::unit_dependencies::deps_of
  11: cargo::core::compiler::unit_dependencies::deps_of
  12: cargo::core::compiler::unit_dependencies::deps_of
  13: cargo::core::compiler::unit_dependencies::build_unit_dependencies
  14: cargo::ops::cargo_compile::create_bcx
  15: cargo::ops::cargo_compile::compile_ws
  16: cargo::ops::cargo_compile::compile
  17: cargo::commands::build::exec
  18: cargo::cli::main
  19: cargo::main