extend the "if-unchanged" logic for compiler builds

[onur-ozkan]

Implements the first item from this tracking issue.

In short, we want to make "if-unchanged" logic to check for changes outside of certain allowed directories, and this PR implements that.

See #131658 for more context.

[rustbot]

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

[onur-ozkan]

I don't want to have this list, but without it, it will be extremely hard to manage RUSTC_IF_UNCHANGED_ALLOWED_PATHS since there will be hundreds of paths 🙁.

[Noratrieb]

I think it really doesn't matter if there are a few more rebuilds than necessary. I'd remove this list here and significantly reduce the list above. I think I'd be fine if it only contained library.

[Noratrieb]

If we have to maintain a list of paths, making it simple and obvious and handle the most important cases seems way more important than making it granular and perfect.

[bjorn3]

Would it make sense to compute the set of files that may not be changed for download-rustc to work based on the .d files in build/host/stage0-rustc? This is what cargo uses for determining if a crate should be recompiled. It doesn't list the Cargo.toml files though. For the aggregated .d file generated by cargo in build/host/stage0-rustc/<host>/release/rustc-main.d I think it would make sense to add Cargo.toml there too. These aggregated .d files are meant for integration with external build systems AFAIK, which need to track Cargo.toml changes too.

[Noratrieb]

this is wrong

[onur-ozkan]

How?

[Noratrieb]

if the dependencies of the compiler are updated, the compiler needs to be rebuilt

[onur-ozkan]

Oh, I didn't consider (but I should) the small bumps that can happen without changing Cargo.toml files in compiler and library tree.

[Noratrieb]

yeah, there are so many things to consider or to not consider here that I think it makes the most sense to be super simple and only special case library as not rebuilding. in case anyone complains we can add more as needed

[onur-ozkan]

@rustbot author

[onur-ozkan]

RUSTC_IF_UNCHANGED_DENIED_SUBPATHS is removed to rely on allowed paths only. Also, "src/bootstrap" is not included in the allowed paths on purpose since it can affect compiler builds.

@rustbot ready

[onur-ozkan]

Would it make sense to compute the set of files that may not be changed for download-rustc to work based on the .d files in build/host/stage0-rustc?

We are comparing against the nightly compiler, "stage0-rustc" wouldn't help.

[bors]

☔ The latest upstream changes (presumably #131934) made this pull request unmergeable. Please resolve the merge conflicts.