Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(resolve): Improve multi-MSRV workspaces #14569

Merged
merged 7 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 54 additions & 20 deletions src/cargo/core/resolver/version_prefs.rs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation-wise looks good. I'd like to hear back from @VorpalBlade for their opinion on this new behavior.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like a step in the right direction, as I understand it, the majority wins, which would solve my issue.

(However, if you go ahead and open a pr in the middle of the night, and then merge it before morning there is little timely feedback I can give. Given timezones, work hours etc it is not reasonable to expect feedback in less than 24 hours.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that was little time for feedback before merging but they noted elsewhere that your feedback didn't have to be blocking for merge as this is unstable. Its still important for stabilization!

Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ pub struct VersionPreferences {
try_to_use: HashSet<PackageId>,
prefer_patch_deps: HashMap<InternedString, HashSet<Dependency>>,
version_ordering: VersionOrdering,
max_rust_version: Option<PartialVersion>,
rust_versions: Vec<PartialVersion>,
}

#[derive(Copy, Clone, Default, PartialEq, Eq, Hash, Debug)]
Expand Down Expand Up @@ -49,8 +49,8 @@ impl VersionPreferences {
self.version_ordering = ordering;
}

pub fn max_rust_version(&mut self, ver: Option<PartialVersion>) {
self.max_rust_version = ver;
pub fn rust_versions(&mut self, vers: Vec<PartialVersion>) {
self.rust_versions = vers;
}

/// Sort (and filter) the given vector of summaries in-place
Expand All @@ -59,7 +59,7 @@ impl VersionPreferences {
///
/// Sort order:
/// 1. Preferred packages
/// 2. [`VersionPreferences::max_rust_version`]
/// 2. Most compatible [`VersionPreferences::rust_versions`]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unstable doc might need an update according, for the algorithm of determining workspace MSRV. However, from a user perspective, it is a bit hard to understand. I am not sure if we want to document it.

It is also hard to determine from a human eye, if it is a large workspace and people are adding/removing members a lot that changes the workspace-wide MSRV.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the text. The unstable docs offer us an opportunity to evaluate the wording. Even once its stabilized, we can iterate as we see how people are dealing with it.

/// 3. `first_version`, falling back to [`VersionPreferences::version_ordering`] when `None`
///
/// Filtering:
Expand All @@ -85,20 +85,11 @@ impl VersionPreferences {
return previous_cmp;
}

if let Some(max_rust_version) = &self.max_rust_version {
let a_is_compat = a
.rust_version()
.map(|a| a.is_compatible_with(max_rust_version))
.unwrap_or(true);
let b_is_compat = b
.rust_version()
.map(|b| b.is_compatible_with(max_rust_version))
.unwrap_or(true);
match (a_is_compat, b_is_compat) {
(true, true) => {} // fallback
(false, false) => {} // fallback
(true, false) => return Ordering::Less,
(false, true) => return Ordering::Greater,
if !self.rust_versions.is_empty() {
let a_compat_count = self.msrv_compat_count(a);
let b_compat_count = self.msrv_compat_count(b);
if b_compat_count != a_compat_count {
return b_compat_count.cmp(&a_compat_count);
}
}

Expand All @@ -112,6 +103,17 @@ impl VersionPreferences {
let _ = summaries.split_off(1);
}
}

fn msrv_compat_count(&self, summary: &Summary) -> usize {
let Some(rust_version) = summary.rust_version() else {
return self.rust_versions.len();
};

self.rust_versions
.iter()
.filter(|max| rust_version.is_compatible_with(max))
.count()
}
}

#[cfg(test)]
Expand Down Expand Up @@ -236,9 +238,9 @@ mod test {
}

#[test]
fn test_max_rust_version() {
fn test_single_rust_version() {
let mut vp = VersionPreferences::default();
vp.max_rust_version(Some("1.50".parse().unwrap()));
vp.rust_versions(vec!["1.50".parse().unwrap()]);

let mut summaries = vec![
summ("foo", "1.2.4", None),
Expand Down Expand Up @@ -267,6 +269,38 @@ mod test {
);
}

#[test]
fn test_multiple_rust_versions() {
let mut vp = VersionPreferences::default();
vp.rust_versions(vec!["1.45".parse().unwrap(), "1.55".parse().unwrap()]);

let mut summaries = vec![
summ("foo", "1.2.4", None),
summ("foo", "1.2.3", Some("1.60")),
summ("foo", "1.2.2", None),
summ("foo", "1.2.1", Some("1.50")),
summ("foo", "1.2.0", None),
summ("foo", "1.1.0", Some("1.40")),
summ("foo", "1.0.9", None),
];

vp.version_ordering(VersionOrdering::MaximumVersionsFirst);
vp.sort_summaries(&mut summaries, None);
assert_eq!(
describe(&summaries),
"foo/1.2.4, foo/1.2.2, foo/1.2.0, foo/1.1.0, foo/1.0.9, foo/1.2.1, foo/1.2.3"
.to_string()
);

vp.version_ordering(VersionOrdering::MinimumVersionsFirst);
vp.sort_summaries(&mut summaries, None);
assert_eq!(
describe(&summaries),
"foo/1.0.9, foo/1.1.0, foo/1.2.0, foo/1.2.2, foo/1.2.4, foo/1.2.1, foo/1.2.3"
.to_string()
);
}

#[test]
fn test_empty_summaries() {
let vp = VersionPreferences::default();
Expand Down
2 changes: 1 addition & 1 deletion src/cargo/core/workspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -664,7 +664,7 @@ impl<'gctx> Workspace<'gctx> {

/// Get the lowest-common denominator `package.rust-version` within the workspace, if specified
/// anywhere
pub fn rust_version(&self) -> Option<&RustVersion> {
pub fn lowest_rust_version(&self) -> Option<&RustVersion> {
self.members().filter_map(|pkg| pkg.rust_version()).min()
}

Expand Down
2 changes: 1 addition & 1 deletion src/cargo/ops/cargo_update.rs
Original file line number Diff line number Diff line change
Expand Up @@ -739,7 +739,7 @@ fn required_rust_version(ws: &Workspace<'_>) -> Option<PartialVersion> {
return None;
}

if let Some(ver) = ws.rust_version() {
if let Some(ver) = ws.lowest_rust_version() {
Some(ver.clone().into_partial())
} else {
let rustc = ws.gctx().load_global_rustc(Some(ws)).ok()?;
Expand Down
2 changes: 1 addition & 1 deletion src/cargo/ops/lockfile.rs
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ pub fn write_pkg_lockfile(ws: &Workspace<'_>, resolve: &mut Resolve) -> CargoRes
// out lock file updates as they're otherwise already updated, and changes
// which don't touch dependencies won't seemingly spuriously update the lock
// file.
let default_version = ResolveVersion::with_rust_version(ws.rust_version());
let default_version = ResolveVersion::with_rust_version(ws.lowest_rust_version());
let current_version = resolve.version();
let next_lockfile_bump = ws.gctx().cli_unstable().next_lockfile_bump;
tracing::debug!("lockfile - current: {current_version:?}, default: {default_version:?}");
Expand Down
2 changes: 1 addition & 1 deletion src/cargo/ops/registry/info/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,6 @@ fn try_get_msrv_from_nearest_manifest_or_ws(
let rust_version = nearest_package.and_then(|p| p.rust_version().map(|v| v.as_partial()));
// If the nearest manifest does not have a specific Rust version, try to get it from the workspace.
rust_version
.or_else(|| ws.and_then(|ws| ws.rust_version().map(|v| v.as_partial())))
.or_else(|| ws.and_then(|ws| ws.lowest_rust_version().map(|v| v.as_partial())))
.cloned()
}
19 changes: 11 additions & 8 deletions src/cargo/ops/resolve.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ use crate::util::errors::CargoResult;
use crate::util::CanonicalUrl;
use anyhow::Context as _;
use cargo_util::paths;
use cargo_util_schemas::core::PartialVersion;
use std::collections::{HashMap, HashSet};
use tracing::{debug, trace};

Expand Down Expand Up @@ -357,14 +358,16 @@ pub fn resolve_with_previous<'gctx>(
version_prefs.version_ordering(VersionOrdering::MinimumVersionsFirst)
}
if ws.resolve_honors_rust_version() {
let rust_version = if let Some(ver) = ws.rust_version() {
ver.clone().into_partial()
} else {
let mut rust_versions: Vec<_> = ws
.members()
.filter_map(|p| p.rust_version().map(|rv| rv.as_partial().clone()))
.collect();
if rust_versions.is_empty() {
let rustc = ws.gctx().load_global_rustc(Some(ws))?;
let rustc_version = rustc.version.clone().into();
rustc_version
};
version_prefs.max_rust_version(Some(rust_version));
let rust_version: PartialVersion = rustc.version.clone().into();
rust_versions.push(rust_version);
}
version_prefs.rust_versions(rust_versions);
}

let avoid_patch_ids = if register_patches {
Expand Down Expand Up @@ -422,7 +425,7 @@ pub fn resolve_with_previous<'gctx>(
&replace,
registry,
&version_prefs,
ResolveVersion::with_rust_version(ws.rust_version()),
ResolveVersion::with_rust_version(ws.lowest_rust_version()),
Some(ws.gctx()),
)?;

Expand Down
4 changes: 3 additions & 1 deletion src/doc/src/reference/unstable.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,9 @@ This was stabilized in 1.79 in [#13608](https://github.com/rust-lang/cargo/pull/
- `package.edition = "2024"` (only in workspace root)

The resolver will prefer dependencies with a `package.rust-version` that is the same or older than your project's MSRV.
Your project's MSRV is determined by taking the lowest `package.rust-version` set among your workspace members.
As the resolver is unable to determine which workspace members will eventually
depend on a package when it is being selected, we prioritize versions based on
how many workspace member MSRVs they are compatible with.
If there is no MSRV set then your toolchain version will be used, allowing it to pick up the toolchain version from pinned in rustup (e.g. `rust-toolchain.toml`).

#### `resolver.incompatible-rust-versions`
Expand Down
66 changes: 46 additions & 20 deletions tests/testsuite/rust_version.rs
Original file line number Diff line number Diff line change
Expand Up @@ -409,22 +409,36 @@ foo v0.0.1 ([ROOT]/foo)

#[cargo_test]
fn resolve_with_multiple_rust_versions() {
Package::new("only-newer", "1.6.0")
Package::new(&format!("shared-only-newer"), "1.65.0")
.rust_version("1.65.0")
.file("src/lib.rs", "fn other_stuff() {}")
.publish();
Package::new("newer-and-older", "1.5.0")
.rust_version("1.45.0")
.file("src/lib.rs", "fn other_stuff() {}")
.publish();
Package::new("newer-and-older", "1.5.1")
.rust_version("1.55.0")
for ver in ["1.45.0", "1.55.0", "1.65.0"] {
Package::new(&format!("shared-newer-and-older"), ver)
.rust_version(ver)
.file("src/lib.rs", "fn other_stuff() {}")
.publish();
}
Package::new(&format!("lower-only-newer"), "1.65.0")
.rust_version("1.65.0")
.file("src/lib.rs", "fn other_stuff() {}")
.publish();
Package::new("newer-and-older", "1.6.0")
for ver in ["1.45.0", "1.55.0"] {
Package::new(&format!("lower-newer-and-older"), ver)
.rust_version(ver)
.file("src/lib.rs", "fn other_stuff() {}")
.publish();
}
Package::new(&format!("higher-only-newer"), "1.65.0")
.rust_version("1.65.0")
.file("src/lib.rs", "fn other_stuff() {}")
.publish();
for ver in ["1.55.0", "1.65.0"] {
Package::new(&format!("higher-newer-and-older"), ver)
.rust_version(ver)
.file("src/lib.rs", "fn other_stuff() {}")
.publish();
}

let p = project()
.file(
Expand All @@ -441,8 +455,10 @@ fn resolve_with_multiple_rust_versions() {
rust-version = "1.60.0"

[dependencies]
only-newer = "1.0.0"
newer-and-older = "1.0.0"
higher-only-newer = "1"
higher-newer-and-older = "1"
shared-only-newer = "1"
shared-newer-and-older = "1"
"#,
)
.file("src/main.rs", "fn main() {}")
Expand All @@ -457,8 +473,10 @@ fn resolve_with_multiple_rust_versions() {
rust-version = "1.50.0"

[dependencies]
only-newer = "1.0.0"
newer-and-older = "1.0.0"
lower-only-newer = "1"
lower-newer-and-older = "1"
shared-only-newer = "1"
shared-newer-and-older = "1"
"#,
)
.file("lower/src/main.rs", "fn main() {}")
Expand All @@ -470,15 +488,17 @@ fn resolve_with_multiple_rust_versions() {
.masquerade_as_nightly_cargo(&["msrv-policy"])
.with_stderr_data(str![[r#"
[UPDATING] `dummy-registry` index
[LOCKING] 2 packages to latest compatible versions
[LOCKING] 6 packages to latest compatible versions

"#]])
.run();
p.cargo("tree")
.with_stdout_data(str![[r#"
higher v0.0.1 ([ROOT]/foo)
├── newer-and-older v1.6.0
└── only-newer v1.6.0
├── higher-newer-and-older v1.65.0
├── higher-only-newer v1.65.0
├── shared-newer-and-older v1.65.0
└── shared-only-newer v1.65.0

"#]])
.run();
Expand All @@ -489,17 +509,23 @@ higher v0.0.1 ([ROOT]/foo)
.masquerade_as_nightly_cargo(&["msrv-policy"])
.with_stderr_data(str![[r#"
[UPDATING] `dummy-registry` index
[LOCKING] 2 packages to latest Rust 1.50.0 compatible versions
[ADDING] newer-and-older v1.5.0 (available: v1.6.0, requires Rust 1.65.0)
[ADDING] only-newer v1.6.0 (requires Rust 1.65.0)
[LOCKING] 6 packages to latest Rust 1.50.0 compatible versions
[ADDING] higher-newer-and-older v1.55.0 (available: v1.65.0, requires Rust 1.65.0)
[ADDING] higher-only-newer v1.65.0 (requires Rust 1.65.0)
[ADDING] lower-newer-and-older v1.45.0 (available: v1.55.0, requires Rust 1.55.0)
[ADDING] lower-only-newer v1.65.0 (requires Rust 1.65.0)
[ADDING] shared-newer-and-older v1.45.0 (available: v1.65.0, requires Rust 1.65.0)
[ADDING] shared-only-newer v1.65.0 (requires Rust 1.65.0)

"#]])
.run();
p.cargo("tree")
.with_stdout_data(str![[r#"
higher v0.0.1 ([ROOT]/foo)
├── newer-and-older v1.5.0
└── only-newer v1.6.0
├── higher-newer-and-older v1.55.0
├── higher-only-newer v1.65.0
├── shared-newer-and-older v1.45.0
└── shared-only-newer v1.65.0

"#]])
.run();
Expand Down
Loading