Tracking issue for -Z features=itarget

[ehuss]

Implementation: #7820
Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#features

Summary
-Z features=itarget causes the feature resolver to ignore features for target-specific dependencies for targets that don't match the current compile target. For example:

[dependency.common]
version = "1.0"
features = ["f1"]

[target.'cfg(windows)'.dependencies.common]
version = "1.0"
features = ["f2"]

When building this example for a non-Windows platform, the f2 feature will not be enabled.

Unresolved issues

• Update cargo metadata
• Does this cause existing projects to fail?
  • Projects can add missing features as a backwards-compatible workaround. However, if it causes problems for too many projects, we may need to find some way to opt-in.

[est31]

Does this affect lockfiles too? As in: if this feature is enabled, will a lockfile on Windows look different from a lockfile on Linux? Or is it only affecting behaviour of the actual build?

[ehuss]

It does not affect Cargo.lock.

[johanlindfors]

This seems to work, at least for my scenario:

[target.'cfg(macos)'.dependencies]
sdl2 = { version = "0.32", features = ["bundled"]}

[target.'cfg(windows)'.dependencies]
sdl2 = { version = "0.32", features = ["bundled", "static-link"]}

Building with: cargo +nightly run -Z features=itarget

But I'm still a bit confused about the proposed solution. This clearly feels like a bug in the current feature resolver, but this fix proposes that we specify a different resolver? Or is this just a temporary solution while this fix gets merged into the "original" resolver?

[ehuss]

@johanlindfors it seems to work for me. What platform are you targeting? What did you see that made you think it isn't working? One thing, cfg(macos) is not a valid expression. It should be cfg(target_os = "macos").

Yes, there is a new resolver, it is not a "bug" per se in the current resolver. The dependency resolver still needs to resolve all platforms for a stable dependency graph. These features also need to be resolved for multiple targets independently (host and target). It may be possible to do it with the current resolver, but I think it would require a multi-pass solution, so I don't think there would be any benefit, and a lot of downsides.

[jdm]

cfg([some target_os value]) is actually a valid cfg expression in current Rust code as far as I can tell. At least cfg(unix) and cfg(windows) seem to work.

[ehuss]

unix and windows are special cases. You can read more about them here. Those are target families, whereas macos is a target os within the unix family.

[jplatte]

@johanlindfors Yes, -Z features=itarget is a temporary solution for trying out the new solver while it is still unstable. It will replace the current solver as the default if things go to plan.

[johanlindfors]

@ehuss You are right, I was using the cfg(target_os = "macos") syntax but when copying the code into the comment on github I manually "shortened" it, thinking I was being smart, turns out, as usual, that was not the case!

:)

[ehuss]

The crater run for this has finished: https://crater-reports.s3.amazonaws.com/pr-69560/index.html

There are about 9 regressions, which is a bit more than I was hoping for:

• https://github.com/FauxFaux/quad-image
• https://github.com/Leinnan/slavic_castles
• https://github.com/chrisabruce/libp2p-ios
• https://github.com/dmweis/lunar_lander
• https://github.com/plaptov/ant_sim_rs
• https://github.com/tomaka/2018-rustrush-demo
• arwa 0.1.0
• label_attribute 0.1.1
• zrs 0.1.5

[benmarten]

I am not able to get this to work:

[target.'cfg(macos)'.dependencies.opencv]
version = "0.32.0"
default-features = false
features = ['opencv-4']

[target.'cfg(linux)'.dependencies.opencv]
version = "0.32.0"
default-features = false
features = ['opencv-32']

cargo build -Z features=itarget

Anything wrong with the syntax?

[ehuss]

@benmarten See above. cfg(macos) is not a valid cfg expression.

[benmarten]

@ehuss
Thanks, that works now on my mac and inside docker/linux.

[target.'cfg(target_os = "macos")'.dependencies.opencv]
version = "0.32.0"
default-features = false
features = ['opencv-4']

[target.'cfg(target_os = "linux")'.dependencies.opencv]
version = "0.32.0"
default-features = false
features = ['opencv-32']

rustup override set nightly
cargo +nightly build -Z features=itarget

[gz]

I'm getting a panic from cargo when enabling the itarget feature and trying to build:

ProcessExecutionError: Command line: ['/home/gz/.cargo/bin/xargo', 'build', '--target', 'x86_64-custom', '--color', 'always', '-Zfeatures=itarget']
Exit code: 101
Stderr:  | warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
         | package:   /home/gz/workspace/besp/lib/node-replication/Cargo.toml
         | workspace: /home/gz/workspace/besp/Cargo.toml
         | thread 'main' panicked at 'features did not find PackageId { name: "libc", version: "0.2.66", source: "registry `https://github.com/rust-lang/crates.io-index`" } false', src/tools/cargo/src/cargo/core/resolver/features.rs:220:17
         | stack backtrace:
         |    0: backtrace::backtrace::libunwind::trace
         |              at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
         |    1: backtrace::backtrace::trace_unsynchronized
         |              at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
         |    2: std::sys_common::backtrace::_print_fmt
         |              at src/libstd/sys_common/backtrace.rs:78
         |    3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
         |              at src/libstd/sys_common/backtrace.rs:59
         |    4: core::fmt::write
         |              at src/libcore/fmt/mod.rs:1069
         |    5: std::io::Write::write_fmt
         |              at src/libstd/io/mod.rs:1427
         |    6: std::sys_common::backtrace::_print
         |              at src/libstd/sys_common/backtrace.rs:62
         |    7: std::sys_common::backtrace::print
         |              at src/libstd/sys_common/backtrace.rs:49
         |    8: std::panicking::default_hook::{{closure}}
         |              at src/libstd/panicking.rs:198
         |    9: std::panicking::default_hook
         |              at src/libstd/panicking.rs:218
         |   10: std::panicking::rust_panic_with_hook
         |              at src/libstd/panicking.rs:511
         |   11: rust_begin_unwind
         |              at src/libstd/panicking.rs:419
         |   12: std::panicking::begin_panic_fmt
         |              at src/libstd/panicking.rs:373
         |   13: cargo::core::resolver::features::ResolvedFeatures::activated_features_int
         |   14: cargo::core::compiler::unit_dependencies::new_unit_dep_with_profile
         |   15: cargo::core::compiler::unit_dependencies::compute_deps
         |   16: cargo::core::compiler::unit_dependencies::deps_of
         |   17: cargo::core::compiler::unit_dependencies::deps_of
         |   18: cargo::core::compiler::unit_dependencies::deps_of
         |   19: cargo::core::compiler::unit_dependencies::deps_of
         |   20: cargo::core::compiler::unit_dependencies::deps_of
         |   21: cargo::core::compiler::unit_dependencies::deps_of
         |   22: cargo::core::compiler::unit_dependencies::deps_of_roots
         |   23: cargo::core::compiler::unit_dependencies::build_unit_dependencies
         |   24: cargo::ops::cargo_compile::compile_ws
         |   25: cargo::ops::cargo_compile::compile
         |   26: cargo::commands::build::exec
         |   27: cargo::cli::main
         |   28: cargo::main
         |   29: std::rt::lang_start::{{closure}}
         |   30: std::rt::lang_start_internal::{{closure}}
         |              at src/libstd/rt.rs:52
         |   31: std::panicking::try::do_call
         |              at src/libstd/panicking.rs:331
         |   32: std::panicking::try
         |              at src/libstd/panicking.rs:274
         |   33: std::panic::catch_unwind
         |              at src/libstd/panic.rs:394
         |   34: std::rt::lang_start_internal
         |              at src/libstd/rt.rs:51
         |   35: main
         |   36: __libc_start_main
         |   37: <unknown>
         | note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

There are no target specific dependencies in the particular lib that is being compiled (node-replication) but there are a few libc dependencies in transitive crates that are declared i.e., like this:

[target.'cfg(unix)'.dependencies]
libc = "0.2"

[ehuss]

@gz Which version are you using? Is this a project I can look at? If not, can you create a reproduction? Or maybe show the dependency tree and the relevant dependency declarations.

Also, it looks like you are using a custom JSON target spec, can you include that, too?

[gz]

@ehuss I put a minimal example here https://github.com/gz/itarget-bug that triggers the panic for me. It seems the culprit is when I try to include the hashbrown dependency.

[ehuss]

@gz TYVM for the repro! I posted a fix at #8048. As a workaround, you can pass -Zfeatures=itarget,host_dep or -Zfeatures=all (requires recent nightly such as nightly-2020-03-27).

[gz]

Thanks for the fix (and the feature, it simplifies my cargo dependencies quite a lot). I can compile my project now with -Zfeatures=all.

[CleanCut]

I'd love to get this on stable. Is this on the path to stabilization?

[zicklag]

Definitely agree this would be great to have, but in case this helps anybody, here's a description of how I got around not having this for gfx:

https://community.amethyst.rs/t/we-need-to-figure-out-a-way-to-adapt-the-features-of-the-amethyst-dependency-to-the-platform-the-game-is-being-compiled-on/1596/6?u=zicklag

[thomwiggers]

It does not seem possible to do

[target.'cfg(not(windows))'.features]
default = ["openssl"]

Is this correct?

[simon-an]

Took me quite some time to find out, that I need to set resolver = "2" in my root Cargo.toml to activate this.
https://doc.rust-lang.org/cargo/reference/features.html

[1sra3l]

Is this the right place to ask about "feature" specific dependencies?

[target.'cfg(features = "fltk")'.dependencies]
optional = true
# specific fltk git repos 

I want a library to expose GUI elements if those features are requested by the end user

[Razican]

Is this the right place to ask about "feature" specific dependencies?

[target.'cfg(features = "fltk")'.dependencies]
optional = true
# specific fltk git repos 

I want a library to expose GUI elements if those features are requested by the end user

You can already do this by adding the dependency in the feature definition, right?

[features]
my_feature = ["my_dependency"]

[dependencies]
my_dependency = { version = "1.0", optional = true }

[1sra3l]

Sorry I was not clear enough, I mean to ONLY included it for the features derived. Similar to compiling with OS specific crates, but only if features are requested. If this is already possible I missed it.
FWIW I thought doing something like

[dependencies]
# things the library needs
[dependencies.my_feature]
# stuff to include ONLY when they want the GUI aspect

There seems like something like this should be possible to me, but it may not be.

[1sra3l]

@Razican I think I misunderstood what I was doing, I think I untangled what I was trying to do... sorry for the clutter 😅