Migrate in-tree crates to 2021 #89103

Mark-Simulacrum · 2021-09-19T17:56:47Z

This replaces #89075 (cherry picking some of the commits from there), and closes #88637 and fixes #89074.

It excludes a migration of the library crates for now (see tidy diff) because we have some pending bugs around macro spans to fix there.

I instrumented bootstrap during the migration to make sure all crates moved from 2018 to 2021 had the compatibility warnings applied first.

Originally, the intent was to support cargo fix --edition within bootstrap, but this proved fairly difficult to pull off. We'd need to architect the check functionality to support running cargo check and cargo fix within the same x.py invocation, and only resetting sysroots on check. Further, it was found that cargo fix doesn't behave too well with "not quite workspaces", such as Clippy which has several crates. Bootstrap runs with --manifest-path ... for all the tools, and this makes cargo fix only attempt migration for that crate. We can't use e.g. --workspace due to needing to maintain sysroots for different phases of compilation appropriately.

It is recommended to skip the mass migration of Cargo.toml's to 2021 for review purposes; you can also use git diff d6cd2c6c877110748296760aefddc21a0ea1d316 -I'^edition = .20...$' to ignore the edition = 2018/21 lines in the diff.

rust-highfive · 2021-09-19T17:56:50Z

Some changes occured to the CTFE / Miri engine

cc @rust-lang/miri

Changes rustc_apfloat. rustc_apfloat is currently in limbo and you almost certainly don't want to change it (see #55993).

cc @eddyb

Some changes occured to rustc_codegen_cranelift

cc @bjorn3

Some changes occurred in src/tools/rustfmt.

cc @calebcartwright

rustdoc-json-types is a public (although nightly-only) API. Consider changing src/librustdoc/json/conversions.rs instead; otherwise, make sure you update format_version.

cc @CraftSpider,@aDotInTheVoid

rust-highfive · 2021-09-19T17:56:51Z

r? @estebank

(rust-highfive has picked a reviewer for you, use r? to override)

Mark-Simulacrum · 2021-09-19T17:58:20Z

The drop time changes which we're accepting as part of this migration could be perf sensitive, but I believe it to be fairly unlikely -- they're mostly not in hot code, and moving the drop slightly shouldn't change too much. Closure sizes are also unlikely to be that important. That said, we should check: @bors try @rust-timer queue

rust-timer · 2021-09-19T17:58:22Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2021-09-19T17:58:29Z

⌛ Trying commit ccb46c236a8bd0eeac370d63a42781750857c3b5 with merge d70706afaa69357a70c0cdfdc13d6de7aa0a4396...

Mark-Simulacrum · 2021-09-19T18:05:33Z

compiler/rustc_interface/src/util.rs

@@ -125,6 +125,7 @@ pub fn scoped_thread<F: FnOnce() -> R + Send, R: Send>(cfg: thread::Builder, f:
    let result_ptr = Ptr(&mut result as *mut _ as *mut ());

    let thread = cfg.spawn(move || {
+        let _ = (&run, &result_ptr);


These are necessary since raw pointers aren't send/sync, but this whole block is likely to go away shortly anyway - #89104

src/tools/tidy/src/edition.rs

bors · 2021-09-19T19:01:07Z

💔 Test failed - checks-actions

camelid · 2021-09-20T02:32:45Z

It looks like some examples in the rust-2021-compatibility lint docs need ```edition2018 added to them.

These were left over in migrations to subtrees, which should generally be treated as-if it was local. Also fixes a warning caused by this change.

This just applies the suggested fixes from the compatibility warnings, leaving any that are in practice spurious in. This is primarily intended to provide a starting point to identify possible fixes to the migrations (e.g., by avoiding spurious warnings). A secondary commit cleans these up where they are false positives (as is true in many of the cases).

estebank · 2021-09-20T15:32:02Z

compiler/rustc_typeck/src/astconv/mod.rs

+                if let GenericParamDefKind::Lifetime = param.kind {
+                    if let hir::GenericArg::Lifetime(lifetime) = &lifetimes[i] {
                        self.ast_region_to_region(lifetime, None).into()
+                    } else {
+                        bug!()
                    }
-                    _ => bug!(),
+                } else {
+                    bug!()


This could have been

match (param.kind, lifetimes.get(i)) { (GenericParamDefKind::Lifetime, Some(hir::GenericArg::Lifetime(lifetime))) => { self.ast_region_to_region(lifetime, None).into() } _ => bug!(), }

but there's no need to change it in this PR :)

estebank · 2021-09-20T15:35:34Z

@bors try @rust-timer queue

r=me with clean bill of perf

rust-timer · 2021-09-20T15:35:35Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2021-09-21T02:23:30Z

⌛ Trying commit 3b89679 with merge f8413f415838486def71cb351f259262f54358cc...

bors · 2021-09-21T03:52:09Z

☀️ Try build successful - checks-actions
Build commit: f8413f415838486def71cb351f259262f54358cc (f8413f415838486def71cb351f259262f54358cc)

rust-timer · 2021-09-21T03:52:11Z

Queued f8413f415838486def71cb351f259262f54358cc with parent 60e70cc, future comparison URL.

rust-timer · 2021-09-21T06:15:29Z

Finished benchmarking commit (f8413f415838486def71cb351f259262f54358cc): comparison url.

Summary: This change led to large relevant mixed results 🤷 in compiler performance.

Large improvement in instruction counts (up to -2.9% on incr-full builds of ctfe-stress-4)
Moderate regression in instruction counts (up to 0.8% on full builds of issue-58319)

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR led to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

estebank · 2021-09-21T18:33:46Z

@bors r+ rollup=never

cc @rust-lang/compiler

Mark-Simulacrum · 2021-09-21T18:33:46Z

We're pretty clearly going to do this migration, and the regression is limited to a small percentage on a single benchmark. I'm going to approve this for now, but not mark it as triaged -- I'd like to make sure we identify the source of the regression, but don't want to block this on that.

@bors r=estebank

bors · 2021-09-21T18:33:47Z

📌 Commit 3b89679 has been approved by estebank

bors · 2021-09-21T18:33:48Z

📌 Commit 3b89679 has been approved by estebank

Mark-Simulacrum · 2021-09-21T18:34:44Z

@bors r=estebank rollup=never (looks like bors lost status over close & reopen)

bors · 2021-09-21T18:34:46Z

💡 This pull request was already approved, no need to approve it again.

There's another pull request that is currently being tested, blocking this pull request: rustc_codegen_llvm: make sse4.2 imply crc32 for LLVM 14 #88981

bors · 2021-09-21T18:34:47Z

📌 Commit 3b89679 has been approved by estebank

bors · 2021-09-21T19:25:52Z

⌛ Testing commit 3b89679 with merge ac2d9fc...

bors · 2021-09-21T22:07:26Z

☀️ Test successful - checks-actions
Approved by: estebank
Pushing ac2d9fc to master...

Migrate in-tree crates to 2021 This replaces rust-lang#89075 (cherry picking some of the commits from there), and closes rust-lang#88637 and fixes rust-lang#89074. It excludes a migration of the library crates for now (see tidy diff) because we have some pending bugs around macro spans to fix there. I instrumented bootstrap during the migration to make sure all crates moved from 2018 to 2021 had the compatibility warnings applied first. Originally, the intent was to support cargo fix --edition within bootstrap, but this proved fairly difficult to pull off. We'd need to architect the check functionality to support running cargo check and cargo fix within the same x.py invocation, and only resetting sysroots on check. Further, it was found that cargo fix doesn't behave too well with "not quite workspaces", such as Clippy which has several crates. Bootstrap runs with --manifest-path ... for all the tools, and this makes cargo fix only attempt migration for that crate. We can't use e.g. --workspace due to needing to maintain sysroots for different phases of compilation appropriately. It is recommended to skip the mass migration of Cargo.toml's to 2021 for review purposes; you can also use `git diff d6cd2c6 -I'^edition = .20...$'` to ignore the edition = 2018/21 lines in the diff.

…Simulacrum tidy: Remove submodules from edition exception list Both style-check and date-check are now on the 2021 edition, and this PR also updates their repositories' submodules. cc rust-lang/rustc-dev-guide#1238 cc rust-lang/reference#1099 cc rust-lang#89103 (comment)

rust-highfive assigned estebank Sep 19, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 19, 2021

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 19, 2021

Mark-Simulacrum mentioned this pull request Sep 19, 2021

Switch rustdoc to Rust 2021 #89075

Closed

Mark-Simulacrum commented Sep 19, 2021

View reviewed changes

camelid reviewed Sep 19, 2021

View reviewed changes

src/tools/tidy/src/edition.rs Show resolved Hide resolved

camelid mentioned this pull request Sep 19, 2021

Recommend rustfmt --edition=2021 rust-lang/rustc-dev-guide#1211

Merged

camelid reviewed Sep 19, 2021

View reviewed changes

src/tools/tidy/src/edition.rs Show resolved Hide resolved

This comment has been minimized.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 19, 2021

Mark-Simulacrum added 3 commits September 20, 2021 08:45

Workaround ICE with if-let and RFC 2229

440d937

Adjust to SourceType::InTree in several places

5e1a614

These were left over in migrations to subtrees, which should generally be treated as-if it was local. Also fixes a warning caused by this change.

Mark-Simulacrum force-pushed the migrate-2021 branch 2 times, most recently from 0eae76e to edab8c7 Compare September 20, 2021 12:48

This comment has been minimized.

Sign in to view

Mark-Simulacrum force-pushed the migrate-2021 branch from edab8c7 to d93dd0c Compare September 20, 2021 13:43

estebank approved these changes Sep 20, 2021

View reviewed changes

rustbot added perf-regression Performance regression. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Sep 21, 2021

estebank closed this Sep 21, 2021

estebank reopened this Sep 21, 2021

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 21, 2021

bors merged commit ac2d9fc into rust-lang:master Sep 21, 2021

rustbot added this to the 1.57.0 milestone Sep 21, 2021

bors mentioned this pull request Sep 21, 2021

Simplify scoped_thread #89104

Merged

camelid mentioned this pull request Oct 21, 2021

tidy: Remove submodules from edition exception list #90143

Merged

jyn514 mentioned this pull request Dec 17, 2021

Update stdlib to the 2021 edition #92030

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate in-tree crates to 2021 #89103

Migrate in-tree crates to 2021 #89103

Mark-Simulacrum commented Sep 19, 2021 •

edited

Loading

rust-highfive commented Sep 19, 2021

rust-highfive commented Sep 19, 2021

Mark-Simulacrum commented Sep 19, 2021

rust-timer commented Sep 19, 2021

bors commented Sep 19, 2021

Mark-Simulacrum Sep 19, 2021

This comment has been minimized.

This comment has been minimized.

bors commented Sep 19, 2021

camelid commented Sep 20, 2021 •

edited

Loading

This comment has been minimized.

estebank Sep 20, 2021

estebank commented Sep 20, 2021

rust-timer commented Sep 20, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

rust-timer commented Sep 21, 2021

rust-timer commented Sep 21, 2021

estebank commented Sep 21, 2021

Mark-Simulacrum commented Sep 21, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

Mark-Simulacrum commented Sep 21, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

Migrate in-tree crates to 2021 #89103

Migrate in-tree crates to 2021 #89103

Conversation

Mark-Simulacrum commented Sep 19, 2021 • edited Loading

rust-highfive commented Sep 19, 2021

rust-highfive commented Sep 19, 2021

Mark-Simulacrum commented Sep 19, 2021

rust-timer commented Sep 19, 2021

bors commented Sep 19, 2021

Mark-Simulacrum Sep 19, 2021

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

bors commented Sep 19, 2021

camelid commented Sep 20, 2021 • edited Loading

This comment has been minimized.

estebank Sep 20, 2021

Choose a reason for hiding this comment

estebank commented Sep 20, 2021

rust-timer commented Sep 20, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

rust-timer commented Sep 21, 2021

rust-timer commented Sep 21, 2021

estebank commented Sep 21, 2021

Mark-Simulacrum commented Sep 21, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

Mark-Simulacrum commented Sep 21, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

bors commented Sep 21, 2021

Mark-Simulacrum commented Sep 19, 2021 •

edited

Loading

camelid commented Sep 20, 2021 •

edited

Loading