Command failing on crate json5

[polarathene]

New to using cargo-minimal-versions, not sure if this is a bug or if I'm using it wrong.

The json5-rs crate has not been updated since Sep 2021. On the config-rs project this dependency was failing with errors in the reproduction output below related to some internal dependencies of the JSON5 crate.

---

Reproduction

[package]
name = "reproduction"
version = "0.1.0"
edition = "2021"

[dependencies]
json5 = "0.4"

# Using Docker to setup reproduction environment
docker run --rm -it rust bash

rustup toolchain add nightly
rustup toolchain add stable
cargo +stable install cargo-hack --locked

curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash

cargo binstall cargo-minimal-versions

cargo minimal-versions check --workspace --ignore-private --detach-path-deps

Output (click to expand)

info: running `rustup run nightly cargo update -Z minimal-versions`
    Updating crates.io index
info: running `cargo hack check --workspace`
info: running `cargo check` on reproduction (1/1)
   Compiling pest_derive v1.0.2
error[E0432]: unresolved import `quote::Ident`
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/lib.rs:251:5
    |
251 | use quote::Ident;
    |     ^^^^^^^^^^^^ no `Ident` in the root
    |
help: consider importing one of these items instead
    |
251 | use ast::Expr::Ident;
    |     ~~~~~~~~~~~~~~~~
251 | use parser::ParserExpr::Ident;
    |     ~~~~~~~~~~~~~~~~~~~~~~~~~
251 | use proc_macro::Ident;
    |     ~~~~~~~~~~~~~~~~~
251 | use proc_macro::TokenTree::Ident;
    |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      and 4 other candidates

error[E0432]: unresolved import `quote::Ident`
  --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/ast.rs:10:5
   |
10 | use quote::Ident;
   |     ^^^^^^^^^^^^ no `Ident` in the root
   |
help: consider importing one of these items instead
   |
10 | use Ident;
   |     ~~~~~
10 | use ast::Expr::Ident;
   |     ~~~~~~~~~~~~~~~~
10 | use parser::ParserExpr::Ident;
   |     ~~~~~~~~~~~~~~~~~~~~~~~~~
10 | use proc_macro::Ident;
   |     ~~~~~~~~~~~~~~~~~
     and 5 other candidates

error[E0432]: unresolved import `quote::Ident`
  --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:12:13
   |
12 | use quote::{Ident, Tokens};
   |             ^^^^^ no `Ident` in the root
   |
   = help: consider importing one of these items instead:
           Ident
           generator::Expr::Ident
           parser::ParserExpr::Ident
           proc_macro::Ident
           proc_macro::TokenTree::Ident
           proc_macro::bridge::Ident
           proc_macro::bridge::TokenTree::Ident
           syn::Ident
           syn::Token::Ident

error[E0432]: unresolved import `quote::Ident`
  --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/parser.rs:18:5
   |
18 | use quote::Ident;
   |     ^^^^^^^^^^^^ no `Ident` in the root
   |
help: consider importing one of these items instead
   |
18 | use Ident;
   |     ~~~~~
18 | use parser::Expr::Ident;
   |     ~~~~~~~~~~~~~~~~~~~
18 | use parser::ParserExpr::Ident;
   |     ~~~~~~~~~~~~~~~~~~~~~~~~~
18 | use proc_macro::Ident;
   |     ~~~~~~~~~~~~~~~~~
     and 5 other candidates

error[E0432]: unresolved import `quote::Ident`
  --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/validator.rs:16:5
   |
16 | use quote::Ident;
   |     ^^^^^^^^^^^^ no `Ident` in the root
   |
help: consider importing one of these items instead
   |
16 | use Ident;
   |     ~~~~~
16 | use ast::Expr::Ident;
   |     ~~~~~~~~~~~~~~~~
16 | use proc_macro::Ident;
   |     ~~~~~~~~~~~~~~~~~
16 | use proc_macro::TokenTree::Ident;
   |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     and 5 other candidates

error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:336:27
    |
336 |             tokens.append(format!("\"{}\"", string));
    |                           ^^^^^^^^^^^^^^^^^^^^^^^^^ expected `&str`, found `String`
    |
    = note: this error originates in the macro `format` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:347:27
    |
347 |             tokens.append(format!("\"{}\"", string));
    |                           ^^^^^^^^^^^^^^^^^^^^^^^^^ expected `&str`, found `String`
    |
    = note: this error originates in the macro `format` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:358:27
    |
358 |             tokens.append(start);
    |                    ------ ^^^^^ expected `&str`, found `String`
    |                    |
    |                    arguments to this method are incorrect
    |
note: method defined here
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/quote-0.3.0/src/tokens.rs:12:12
    |
12  |     pub fn append(&mut self, token: &str) {
    |            ^^^^^^
help: consider borrowing here
    |
358 |             tokens.append(&start);
    |                           +

error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:360:27
    |
360 |             tokens.append(end);
    |                    ------ ^^^ expected `&str`, found `String`
    |                    |
    |                    arguments to this method are incorrect
    |
note: method defined here
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/quote-0.3.0/src/tokens.rs:12:12
    |
12  |     pub fn append(&mut self, token: &str) {
    |            ^^^^^^
help: consider borrowing here
    |
360 |             tokens.append(&end);
    |                           +

error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:511:27
    |
511 |             tokens.append(format!("\"{}\"", string));
    |                           ^^^^^^^^^^^^^^^^^^^^^^^^^ expected `&str`, found `String`
    |
    = note: this error originates in the macro `format` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:522:27
    |
522 |             tokens.append(format!("\"{}\"", string));
    |                           ^^^^^^^^^^^^^^^^^^^^^^^^^ expected `&str`, found `String`
    |
    = note: this error originates in the macro `format` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:533:27
    |
533 |             tokens.append(start);
    |                    ------ ^^^^^ expected `&str`, found `String`
    |                    |
    |                    arguments to this method are incorrect
    |
note: method defined here
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/quote-0.3.0/src/tokens.rs:12:12
    |
12  |     pub fn append(&mut self, token: &str) {
    |            ^^^^^^
help: consider borrowing here
    |
533 |             tokens.append(&start);
    |                           +

error[E0308]: mismatched types
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/generator.rs:535:27
    |
535 |             tokens.append(end);
    |                    ------ ^^^ expected `&str`, found `String`
    |                    |
    |                    arguments to this method are incorrect
    |
note: method defined here
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/quote-0.3.0/src/tokens.rs:12:12
    |
12  |     pub fn append(&mut self, token: &str) {
    |            ^^^^^^
help: consider borrowing here
    |
535 |             tokens.append(&end);
    |                           +

error[E0599]: no method named `as_ref` found for struct `quote::Tokens` in the current scope
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pest_derive-1.0.2/src/lib.rs:319:15
    |
319 |     generated.as_ref().parse().unwrap()
    |               ^^^^^^ method not found in `Tokens`

Some errors have detailed explanations: E0308, E0432, E0599.
For more information about an error, try `rustc --explain E0308`.
error: could not compile `pest_derive` (lib) due to 14 previous errors
error: process didn't exit successfully: `/usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo check --manifest-path Cargo.toml` (exit status: 101)
error: process didn't exit successfully: `/usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo hack check --workspace` (exit status: 1)

---

If I run cargo check prior to this, it will work just fine. Afterwards a cargo clean will restore that, so it seems to be due to the downgrade / adjustments by cargo-minimal-versions?

Remove/Add/Downgrade output

info: running `rustup run nightly cargo update -Z minimal-versions`
    Updating crates.io index
    Removing block-buffer v0.10.4
    Removing cfg-if v1.0.0
    Removing cpufeatures v0.2.11
    Removing crypto-common v0.1.6
    Removing digest v0.10.7
    Removing generic-array v0.14.7
 Downgrading json5 v0.4.1 -> v0.4.0
    Removing libc v0.2.149
      Adding maplit v1.0.0
    Removing once_cell v1.17.2
    Removing pest v2.5.7
      Adding pest v1.0.0
      Adding pest v2.0.0
    Removing pest_derive v2.5.7
      Adding pest_derive v1.0.2
      Adding pest_derive v2.0.0
    Removing pest_generator v2.5.7
 Downgrading pest_meta v2.5.7 -> v2.0.0
 Downgrading proc-macro2 v1.0.69 -> v0.4.4
    Removing quote v1.0.33
      Adding quote v0.3.0
      Adding quote v0.6.3
 Downgrading serde v1.0.190 -> v1.0.0
    Removing serde_derive v1.0.190
    Removing sha2 v0.10.8
    Removing syn v2.0.38
      Adding syn v0.11.0
      Adding syn v0.14.1
    Removing thiserror v1.0.50
    Removing thiserror-impl v1.0.50
    Removing typenum v1.17.0
 Downgrading ucd-trie v0.1.6 -> v0.1.1
    Removing unicode-ident v1.0.12
      Adding unicode-xid v0.0.3
      Adding unicode-xid v0.1.0
    Removing version_check v0.9.4

Not sure why it removes the dependency and adds two different versions? The failures are happening from the lower version added?

    Removing pest v2.5.7
      Adding pest v1.0.0
      Adding pest v2.0.0

...

    Removing quote v1.0.33
      Adding quote v0.3.0
      Adding quote v0.6.3

[taiki-e]

From a cargo-minimal-versions perspective, this is working correctly. (cargo-minimal-versions detected the issues in json5's Cargo.toml.)

json5's Cargo.toml specifies past = "2.0", pest_derive = "2.0" (which mean pase = ">= 2.0.0", pest_derive = ">= 2.0.0"), but in fact, an older version of past_derive cannot be compiled with the minimal version of its dependencies.

As for the pest_derive requirement, it could be fixed by adding pest_derive = "2.5.7".

Additionally, the serde requirement in json5 also points to an older version (serde = "1.0", which means serde = ">= 1.0.0") that it cannot really compile.

error: cannot find macro `forward_to_deserialize_any_helper` in this scope
   --> /Users/taiki/.cargo/registry/src/index.crates.io-6f17d22bba15001f/json5-0.4.0/src/de.rs:245:5
    |
245 | /     forward_to_deserialize_any! {
246 | |         bool char str string bytes byte_buf unit unit_struct seq
247 | |         tuple tuple_struct map struct identifier ignored_any
248 | |     }
    | |_____^ help: a macro with a similar name exists: `forward_to_deserialize_any`

As for the serde requirement, it could be fixed by adding serde = "1.0.103". (Actually, a slightly older version would work, but AFAIK 1.0.103 is the first minimal-versions-compatible serde version.)

I confirmed that it is now compiled by adding the following:

 [dependencies]
 json5 = "0.4"
+pest_derive = "2.5.7"
+serde = "1.0.103"

(Ideally, the json5's Cargo.toml should be updated, but this can also be addressed by specifying the appropriate dependency requirements on the user side.)

[polarathene]

Additionally, the serde requirement in json5 also points to an older version (serde = "1.0", which means serde = ">= 1.0.0") that it cannot really compile.

My understanding of serde = "1.0" was equivalent to semver ^1.0.0? Did you mean it resolves to a newer version of serde above 1.0.103? (EDIT: Right, you're proving that it cannot compile below 1.0.103 💡 )

As for the serde requirement, it could be fixed by adding serde = "1.0.103". (Actually, a slightly older version would work, but AFAIK 1.0.103 is the first minimal-versions-compatible serde version.)

How do you go about determining that from the failure? Is cargo-minimal-versions able to assist with that? Or only detecting the crates semver for serde is not actually the minimum compatible version?

---

Ideally, the json5's Cargo.toml should be updated

Yes, but the crate no longer seems to be actively maintained so that's unlikely 😅

---

Ok so cargo-minimal-versions is used to (as per -Z minimal-versions, but working around issues):

• Verify if it's actually possible to build with the minimal version from the Cargo.toml semver range of dependencies?
• When it can't, a failure like above is expected and you need to manually identify the actual minimal version range to update Cargo.toml with? (if no Cargo.lock exists, it'll have the minimal version stored there)

For dependencies outside of the project that other crates are responsible for, min version changes need to be contributed upstream, or the project needs to additionally maintain the minimal version of implicit dependencies in Cargo.toml?

-Z direct-minimal-versions may be a better option sometimes? (minimal-versions notes that it's intended to take this approach in future?)

---

Determining dependency versions that are compatible at resolving within MSRV

This is probably due to my inexperience with this, so I've collapsed most of it but sharing in case it helps anyone else.

Not sure if this is something cargo-minimal-versions would be a good fit for, it seems that -Z msrv-policy will eventually work well once rust-version is adopted in Cargo.toml more.

Resolved (out of scope for `cargo-minimal-versions`)

I manually ran cargo-minimal-versions adding each dependency of config-rs (0.13.x release branch) and commented out those that failed.

notify was a bit interesting, it was passing by itself, but not when any other dependency was added 🤔 (an error about mem::transmute) (v4 was released Jan 2020 vs tokio 1.0.1 at Dec 2020)

error[E0591]: can't transmute zero-sized type
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/nix-0.4.2/src/sched.rs:168:20
    |
168 |         ffi::clone(mem::transmute(callback), ptr as *mut c_void, flags, &mut cb)
    |                    ^^^^^^^^^^^^^^
    |
    = note: source type: for<'a> extern "C" fn(*mut Box<(dyn FnMut() -> isize + 'a)>) -> i32 {callback}
    = note: target type: *const for<'a> extern "C" fn(*const Box<(dyn FnMut() -> isize + 'a)>) -> i32
    = help: cast with `as` to a pointer instead

That error was also encountered with cargo +nightly check -Z minimal-versions but not cargo +nightly check -Z direct-minimal-versions, not sure if cargo-minimal-versions supports direct-minimal-versions? (EDIT: Nope not yet, but there's a tracking issue)

[dependencies]
serde = "1"
serde_derive = "1"
rust-ini = "0.18"
glob = "0.3"
temp-env = "0.2"
lazy_static = "1"
nom = "7"
float-cmp = "0.9"
futures = "0.3"
chrono = { version = "0.4", features = ["serde"] }
pathdiff = "0.2"
toml = "0.5"
yaml-rust = "0.4"
serde_json = "1"
indexmap = { version = "1.7", features = ["serde-1"] }
tokio = { version = "1", features = ["rt-multi-thread", "macros", "fs", "io-util", "time"] }

# Fails with any other dependency?
#notify ="4"

# None of these independently pass cargo-minimal-versions
#ron = "0.7"
#json5 = "0.4"
#reqwest = "0.11"
#warp = "=0.3.1"
#async-trait = "0.1"

---

I was curious if cargo-minimal-versions could be used to determine the MSRV minimal compatible version, such as with tokio where the 1.30 release bumped MSRV to 1.63:

[package]
name = "msrv-example"
version = "0.1.0"
edition = "2021"

[dependencies]
tokio = "1"

# MSRV of config-rs in `0.13.x` releases:
$ cargo +1.56.1 check
    Updating crates.io index
error: package `tokio v1.33.0` cannot be built because it requires rustc 1.63 or newer, while the currently active rustc version is 1.56.1

$ cargo minimal-versions check
info: running `rustup run nightly cargo update -Z minimal-versions`
    Updating crates.io index
info: running `cargo hack check`
info: running `cargo check` on msrv-example (1/1)

$ grep -A1 'name = "tokio"' Cargo.lock
name = "tokio"
version = "1.0.1"

cargo-minimal-versions identifies 1.0.1 as the minimal version, but that resolves to 1.33 (if no Cargo.lock exists, and locking to 1.0.1 would be a bit silly?) which would fail.

I'm aware of a separate project, cargo-msrv that will identify / verify an MSRV based on the Cargo.toml dependencies passing a predicate like cargo check for a given toolchain. I thought cargo-minimal-versions could compliment that, but I think that's not the case?

A contributor at config-rs wanted to identify proper minimal versions that would be compatible with the project MSRV, or rather identifying the latest version that still complies with the projects MSRV so bugs fixes and security patches are still valid.

UPDATE: Seems I was after cargo +nightly generate-lockfile -Z msrv-policy (as cargo-minimal-versions derives a minimum version intended for Cargo.toml, not Cargo.lock), which can generate a Cargo.lock using the latest crate version that satisfies semver and is compatible to the MSRV.

Resolved (`-Z msrv-policy` relies on Crates declaring `rust-version` to be reliable)

cargo +1.56.1 check however would still fail:

$ cargo +1.56.1 check
    Updating crates.io index
error: failed to select a version for the requirement `memchr = "=2.6.0"`
candidate versions found which didn't match: 2.5.0, 2.4.1, 2.4.0, ...
location searched: crates.io index
required by package `object v0.31.1`
    ... which satisfies dependency `object = "=0.31.1"` of package `backtrace v0.3.68`
    ... which satisfies dependency `backtrace = "=0.3.68"` of package `tokio v1.29.1`
    ... which satisfies dependency `tokio = "=1.29.1"` of package `msrv-example v0.1.0 (/tmp)`

cargo-msrv --min 1.55.0 --max 1.63.0 would identify that the MSRV is 1.60.0, thus -Z msrv-policy isn't exactly accurate? Nor does it seem I can use it in combination with cargo-minimal-versions 🤷

I can manually decrement until I get something compatible. I was just curious if there was tooling to help automate figuring out what those versions are 😅

object 0.31.1 has the dependency memchr = "2.4.0", which can resolve to 2.6.0 of memchr that introduced rust-version = "1.60.0 with 2.6.2.

That appears to be due to 2.6.0 being assumed as valid to -Z msrv-policy for the MSRV. While tokio 1.29.1 had already declared an MSRV of 1.56.0. Which also explains why the tokio dependency on backtrace 0.3.58 was resolved to 0.3.68 (the version prior to declaring it's own rust-version). Mystery solved 🎉

So like your earlier response, this approach works provided I ensure the lockfile (for CI MSRV checks) stays below memchr 2.6.0:

• Update Cargo.lock: cargo update --package memchr --precise 2.5.0
• Or Cargo.toml: `memchr = "~2.5.0"

---

Resolved (`-Z msrv-policy` bumping incompatible implicit dependencies due to no compatible `rust-version` declared)

The advice to add dependencies to Cargo.toml doesn't seem to work too well with rust-ini = "0.18:

$ cargo +nightly update -Z msrv-policy
    Updating crates.io index
    Updating ahash v0.4.8 -> v0.7.7
    Updating dlv-list v0.2.3 -> v0.3.0
    Updating hashbrown v0.9.1 -> v0.12.3
      Adding once_cell v1.18.0
    Updating ordered-multimap v0.4.0 -> v0.4.3
    Removing ppv-lite86 v0.2.17
    Removing rand v0.8.5
    Removing rand_chacha v0.3.1
    Removing rand_core v0.6.4
      Adding version_check v0.9.4

$ cargo +1.56.1 check
    Updating crates.io index
error: failed to select a version for the requirement `ahash = "=0.7.7"`
candidate versions found which didn't match: 0.4.8, 0.4.1, 0.4.0, ...
location searched: crates.io index
required by package `hashbrown v0.12.3`
    ... which satisfies dependency `hashbrown = "=0.12.3"` of package `ordered-multimap v0.4.3`
    ... which satisfies dependency `ordered-multimap = "=0.4.3"` of package `rust-ini v0.18.0`
    ... which satisfies dependency `rust-ini = "=0.18.0"` of package `msrv-example v0.1.0 (/tmp)`

The ahash crate yanked versions recently. I'm not sure if the 0.6.x series was compatible with 1.56.1, while 0.7.x seems to be 1.60.0 MSRV, there is 0.4.x that is compatible, while hashbrown 0.9.1 refers to 0.6.1 before minor releases after that bump minor versions of ahash.

ordered-multimap is placing a hard dependency on hashbrown? Non-issue building the rust-ini 0.18 crate with Rust 1.56.1 so long as -Z msrv-policy isn't used with it due to the bumping of internal dependencies 🤔 So the answer in this case is that the ordered-multimap dependency of rust-ini needs to not be bumped 🤷‍♂️ (that works)

Problem this time was due to overriding the dependency chain partially, but since there wasn't a compatibility issue prior, I just needed to work in reverse instead reverting the rust-ini dependency ordered-multimap 👍

[taiki-e]

Support for -Z direct-minimal-versions has been added (as --direct flag) in 0.1.21 (#25). It would also be reasonable to use it in your case where the dependencies are incompatible with -Z minimal-versions.

[polarathene]

Support for -Z direct-minimal-versions has been added (as --direct flag) in 0.1.21

Fantastic! Thanks for that ❤️

Does cargo-minimal-versions also offer a benefit of this option vs using it directly via nightly? Or would it function the same unlike the -Z minimal-versions?

My interest was just to verify that a library can build in CI while meeting it's stated MSRV / rust-version, ideally without requiring much messing around with cargo update --package --precise.

I suppose that can still happen with --direct if the implicit dependencies semver resolves to something incompatible on the upper bound. Once rust-version is adopted more, -Z msrv-policy should hopefully introduce the least amount of friction (at least once there's a common baseline rust-version established).

[taiki-e]

Does cargo-minimal-versions also offer a benefit of this option vs using it directly via nightly? Or would it function the same unlike the -Z minimal-versions?

cargo-minimal-versions can be used to catch some issues that are missed when using -Zminimal-versions directly, such as dev-dependencies handling, path dependencies handling, and workspace handling. (See details section in readme for more)
Some of those issues still exist in -Zdirect-minimal-versions.