Cargo report future-incompat

[pnkfelix]

rendered

Summary: Cargo should alert developers to upstream dependencies that trigger future-incompatibility warnings. Cargo should list such dependencies even when these warnings have been suppressed (e.g. via --cap-lints or #[allow(..)] attributes.)

Cargo could additionally provide feedback for tactics a maintainer of the downstream crate could use to address the problem (the details of such tactics is not specified nor mandated by this RFC).

Pre-RFC internals thread

[ehuss]

This seems to boil down to "how annoying is an un-silencable warning". This seems to be a pretty decent balance, though it is tough to predict how people will react.

We could try conservative steps to see how it goes. For example, only turn it on for 1 lint, and see how long it takes for packages on crates.io to update.

Another possibility is to have a policy of a grace period when a future-incompatible lint is first introduced, leave it as capable for 1 Rust release, and then make it un-capable. This would give 6 weeks for opportunistic updates before forcing the warning on everyone. I don't know if that is really necessary, though, or what the typical timeline will look like. I assume existing lints have already been around for a while.

[nrc]

IMO, for most users, most of the time this would be useless and therefore annoying - the only way to exert pressure upstream is submitting a PR and most users can't or won't do that. Where this might be useful is where the maintainer of the downstream crate also maintains the upstream crate.

The Cargo team just discussed the PR. We think this would be nice to have (especially if @pnkfelix can implement), but not with every build. I.e., we should rate limit in some way. Some suggestions from the Cargo team:

• Do it every time as proposed.
• Only warn when Cargo is updated (i.e., rustup update).
• Do it every time but only from a given time (either a fixed time before the warning is intended to be made into an error or manually specified by the compiler team).
• Ramp up warnings on a schedule towards the time when the warning becomes an error (e.g., no warning at first, warnings once per week 6 weeks before, then every time 2 weeks before).

Once we decide on the warning schedule then we can accept the RFC and begin implementation with the caveat that we would intend to iterate on the frequency of warnings in the future as they are found to not have the desired effect or be too annoying.

[shepmaster]

the only way to exert pressure upstream is submitting a PR

Not to detract from the rest of your points, but why is this the only way? Issues, mentioning the author in forum posts / social media, etc. are all ways to encourage a library to update.

[pnkfelix]

@nrc wrote:

Ramp up warnings on a schedule towards the time when the warning becomes an error (e.g., no warning at first, warnings once per week 6 weeks before, then every time 2 weeks before).

Ooh, I like this telescoping warning frequency.

Of course, doing this would exacerbate the problem I mentioned in the RFC text when I covered the annoyance as a drawback:

• (Perhaps we could restrict it so that it only gets re-emitted once each day or some other way to modulate the annoyance level; but of course then the diagnostic would have to state very clearly that it is doing that strange magic.)

But I really like this approach, especially the idea of tying the frequency to when the lint is actually going to become a hard error. It seems like right balance to strike.

And as @nrc says, the details of the actual frequency is a policy matter that can be resolved later, in tandem with implementation.

I will update the RFC text to incorporate annoyance modulation into the system. (I personally would like to expose knobs to let the end-user tweak this, but that is yet another detail that can be resolved later.)

[Diggsey]

It would be useful if there was a way to generate a "future-incompat" report for use in CI/integrations with GitHub etc.

Does cargo currently have any kind of telemetry?

[nrc]

but why is this the only way?

I should have said something like "only effective way" - unless the maintainer is just unaware of the change (which seems unlikely if the crate is maintained) then I think either the maintainer will eagerly correct the issue, or won't correct the issue no matter how many issues are filed or complaints made (apologies for the end of year cynicism :-) ).

[dekellum]

RUSTFLAGS=-Dwarnings is a useful setting for CI, as used in several projects. Does anyone know (or should this RFC address) how this would interact? I couldn't find a mention. Right now, setting that ENV variable does not cause any (--cap-lints applicable) warnings in dependencies to be denied (or shown for that matter).

[est31]

@nrc

unless the maintainer is just unaware of the change (which seems unlikely if the crate is maintained)

I maintain a bunch of crates. Most of those do not get regular active updates. I don't run the compiler on all my crates every time a new rustc is released to check for new future incompat links, and I don't want to do it. I'd claim that most of crates.io crates are maintained that way.

won't correct the issue no matter how many issues are filed or complaints made (apologies for the end of year cynicism :-) ).

Yeah some maintainers can be stubborn. Then it's especially valuable to have downstream be informed about the situation, as the upstream maintainer dangers their build. People can switch away for example, fork the crate, etc.

Ramp up warnings on a schedule towards the time when the warning becomes an error (e.g., no warning at first, warnings once per week 6 weeks before, then every time 2 weeks before).

In fact I think this would lead to more annoyance, not less. If it's always warning, you'll encounter it when you update your rustc or introduce a crate to your dependencies either through an update or an addition to a Cargo.toml. You know the warning is there and can blend it out. If it appears out of nowhere to you while you are doing something completely unrelated, you will be far more annoyed. You'll be even more annoyed if since then, you have closed the console, overflown the buffer or you gotten a report by a user but can't reproduce it because the report was done on a different day and now the user can't tell you the precise crate with the problem either.

And after you committed a fix, how are you sure it's actually a fix to your issue? Consider you don't want any dependencies that create such warnings in your tree. How do you review PRs for them?

TLDR: I don't think that volatile warnings based on time, a random variable or such would be good ideas.

I will update the RFC text to incorporate annoyance modulation into the system. (I personally would like to expose knobs to let the end-user tweak this, but that is yet another detail that can be resolved later.)

Knobs to always turn it on would be very useful. They could only live in the CI of a project to prevent introduction of problematic crates.

[pnkfelix]

And after you committed a fix, how are you sure it's actually a fix to your issue? Consider you don't want any dependencies that create such warnings in your tree. How do you review PRs for them?

This is a compelling argument to me that the default setting should be what I described in the RFC, while @nrc's argument still convinces me that users should be given some form of an annoyance modulation knob.

[pnkfelix]

RUSTFLAGS=-Dwarnings is a useful setting for CI, as used in several projects. Does anyone know (or should this RFC address) how this would interact? I couldn't find a mention. Right now, setting that ENV variable does not cause any (--cap-lints applicable) warnings in dependencies to be denied (or shown for that matter).

I would think that this is an orthogonal issue/bug from the one described here?

[pnkfelix]

I will update the RFC text to incorporate annoyance modulation into the system. (I personally would like to expose knobs to let the end-user tweak this, but that is yet another detail that can be resolved later.)

Okay, commit 7bebac8 is my attempt to resolve this (hopefully last?) remaining point.

I always relish any opportunity to make absurdly subtle references to my past jobs

[ehuss]

@rfcbot reviewed

[Eh2406]

@rfcbot reviewed

[pnkfelix]

ping @Zoxc @estebank @michaelwoerister @petrochenkov regarding checkboxes

[Centril]

Oh so I should also mention that I'm assuming a similar transitionary role that happens today, which looks something like:

• Behavior is changed, crates says impact is huge, warning is implemented instead

• Time passes, some authors see the warning, some crates are updated

• Eventually another rust-lang/rust PR attempts to make the warning a hard error, crater still says there's lots of breakage

• rust-lang/rust authors proactively send PRs to core broken crates. Some changes are merged

• this is when the future incompat lint is turned on, literally everyone gets warnings

• More crates are published and more builds are warning-free

I think it's true that taken literally it's a bit too strict to require something actionable. I feel though that when future incompat lints are turned on is late enough in the process that your action item is very likely to either be cargo update or "go ping the PR and ask for a merge"

I don't really mind one way or another whether it's in the RFC itself, but I think it may be worthwhile to perhaps write a bit about when we expect future incompat lints to get turned on. For example the idea that given a warning it's "at least somewhat actionable", with examples such as cargo update or "go poke an already-existing PR" or something like that may be good to have.

I do not think this is the place to decide on such a policy and that this should be left to the language and compiler teams to tweak things as necessary. I also don't agree with the guidance proposed. It is wildly more conservative than was discussed originally regarding when the infrastructure in rust-lang/rust#59658 would be activated. Imposing these requirements also doesn't seem justifiable by RFC 1122. There are a number of problems with it that render the guidance quite counter productive to fixing bugs in the compiler:

• It moves almost all of the burden on fixing the bugs in the crates to the compiler and language teams and the PR author which do not have these resources to file PRs actively to all the crates when they are numerous. I think it's also unfair to centralize the (mostly unglamorous, repetitive, and boring) work to a few people who mostly do free work and don't have time rather than to distribute the work to more people.

• It's not applicable (or not practically applicable) in cases like Tracking issue for future-incompatbility lint order_dependent_trait_objects rust#56484 where the author of the traitobject crate has disappeared and no attempt to reach them via PRs, emails, etc. has been productive. The only fix here is to fork the crate.

• Because it places heavy burdens on fixing the bugs the practical effect is that the compiler team doesn't triage these bugs to drive fixing them to completion and so the bugs won't be fixed.

[steveklabnik]

It moves almost all of the burden on fixing the bugs in the crates to the compiler and language teams and the PR author which do not have these resources to file PRs actively to all the crates when they are numerous.

One may argue that this is exactly the right alignment of incentives: those who are proposing things that may break should feel the pain that they may break. That this disincentives them to make said changes is a positive, not a negative.

I think it's also unfair to centralize the (mostly unglamorous, repetitive, and boring) work to a few people who mostly do free work and don't have time rather than to distribute the work to more people.

It could also be considered unfair that the community bears the burden of unglamorous, repetitive, and boring work, simply because a team would rather not do it, yet wants to make a change anyway.

[Centril]

One may argue that this is exactly the right alignment of incentives: those who are proposing things that may break should feel the pain that they may break. That this disincentives them to make said changes is a positive, not a negative.

I think it's definitely not a positive to dis-incentivize fixing bugs (and worse yet, soundness holes when those occurs) in the compiler. This also increases the requirements to get things right when shipping and the speed at which we can make improvements if we then impose a lot of constraints when mistakes are made in implementation. Dis-incentivizing making the change in the first place, particularly as it relates to soundness holes is I think counter to the spirit (and text) of RFC 1122 and to the notion that Rust is a safe language (which is more important than backwards compatibility).

It could also be considered unfair that the community bears the burden of unglamorous, repetitive, and boring work, simply because a team would rather not do it, yet wants to make a change anyway.

It's not repetitive because it's distributed to several people who might only have to do a small change in a single crate as opposed to the team or the PR author who would have to do the same fix over and over again (particularly if the regression is large). So no, I do not agree that it is an unfair burden when any individual person's burden is small. (It's even less unfair when large companies are involved.)

[pnkfelix]

@Centril @steveklabnik

Just to keep things clear, at least in my own head:

• I infer that @Centril 's remarks are focused on responding to the protocol outlined in @alexcrichton 's comment from January 10th. (I think this inference is reasonable since @Centril quoted the aforementioned comment.)
• So, do either of you take issue with the (significantly less specific, deliberately underspecified) text that is actually in the RFC text, namely that in the policy issues section?
• I think the text there is at least compatible with what @Centril has said... and maybe also what @steveklabnik has said as well. It leaves the definition and/or tweaking of policy in the hands of the relevant teams, and the only "prescriptive" notes are couched in conditional language ("if our primary goal is X, then we should follow path Y") ?

[Centril]

@pnkfelix

1. The inference is correct. :)
2. I do take issue with:

If our primary goal is to minimize user frustration with our tools and ecosystem, then future-incompatibility reporting for a given lint should be turned only after much of the crate ecosystem have new fixed versions. In other words, we should strive for a steady-state where the typical user response to a future-incompatibility report is that user then runs cargo update, or they ask for a (pre-existing) PR to be merged.

I think this nudges us towards a policy that is more conservative than today, whereas I think we need to move in the opposite direction as the compiler team has failed to actually triage the C-future-compatibility bugs as e.g. noted in:

At the beginning of each 6-week release cycle, the Rust compiler team will review the set of outstanding future compatibility warnings and nominate some of them for Final Comment Period. Toward the end of the cycle, we will review any comments and make a final determination whether to convert the warning into a hard error or remove it entirely.

and generally does not invest the time into fixing the not-adhering-to-the-spec-spec bugs.

3. But it is conditional, as you mention, so perhaps since there is disagreement re. X, then Y won't happen? I don't know what to make of that, as it sorta still implies that X is true, and so it implies a shift in sentiment towards a more conservative breakage policy than I believe is the case today.

[shepmaster]

Taking a page from the JavaScript world, if Rust had a "codemod" / "codeshift" like facility, that might lower the burden of certain types of large scale changes, instead of moving the burden around.

The Rust team could prepare the transformation and apply it against the compiler and "key" crates verifying it works, then allow the long tail of affected people to apply it to their code. The team could submit PRs for "appropriate" crates as well.

It could even potentially be integrated with crater to find out how successful a proposed fix is in the wild.

[jplatte]

There's rerast, but I don't know how well it can handle existing future-incompat things.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[eddyb]

@shepmaster How far off is rustfix? (working off suggestions emitted by rustc)

My understanding is that those JS projects are effectively compiler frontends (or even more of a compiler), whereas rustc includes suggestions in its diagnostics.

IIRC, as part of Rust 2018, paths (e.g. in use) at the very least were rustfix-able.

[pnkfelix]

@eddyb my inference is that, at least historically, the scope of rustfix was meant to be language edition migration, not migration between versions of the compiler.

[pnkfelix]

@eddyb my inference is that, at least historically, the scope of rustfix was meant to be language edition migration, not migration between versions of the compiler.

(but maybe your comment is that we should expand the scope of rustfix to include addressing future-incompatibilities...?)

[ehuss]

the scope of rustfix was meant to be language edition migration

I think it is used for much more than edition migrations. MachineApplicable suggestions are used in everyday development. Several editors provide one-click auto-fix suggestions, and some developers just run cargo fix to bulk apply suggestions.

[pnkfelix]

the scope of rustfix was meant to be language edition migration

I think it is used for much more than edition migrations. MachineApplicable suggestions are used in everyday development. Several editors provide one-click auto-fix suggestions, and some developers just run cargo fix to bulk apply suggestions.

my mistake then! I should have done more research before responding

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

[Aaron1011]

It looks like the FCP finished - should this RFC be merged?

[CAD97]

ping @rust-lang/rfc-pushers why hasn't this been merged yet?

[pnkfelix]

Huzzah! The @rust-lang/compiler and @rust-lang/cargo teams have decided to accept this RFC.

To track further discussion, subscribe to the tracking issue here: rust-lang/rust#71249