x.py: run `rustup toolchain link` in setup #89212

Sl1mb0 · 2021-09-24T00:38:16Z

Addresses #89206

src/bootstrap/setup.rs

jyn514 · 2021-09-24T01:34:06Z

src/bootstrap/setup.rs

+    let build = TargetSelection::from_user(&env!("BUILD_TRIPLE"));
+    let stage_path = ["build", build.rustc_target_arg(), "stage1"].join("/");
+    let toolchain_link =
+        Command::new("rustup").args(&["toolchain", "link", "stage1", &stage_path[..]]).output();


I just realized this will overwrite any existing toolchain - that seems kind of not great, it means we'll delete any existing toolchain with this name. @kinnison @rbtcollins is that intentional that it will silently overwrite?

I guess to avoid that we could explicitly run rustup toolchain list and see if it already has stage1.

Usually this command is executed by the user, so like most Unix-y commands, dispatches as silently and swiftly as possible, and alerting someone is a sign of failure.

Hmm - but ln -s (which you said in the original issue is basically what toolchain link does) gives an error if the file already exists:

$ ln -s x config.toml ln: failed to create symbolic link 'config.toml': File exists

Reasonable~
But the other reason I mentioned links is due to problems due to OS control over when files may be edited/clobbered/removed, which has less to do with the particular link.

Since we have to create the directory in the first place, couldn't we just not link if it already exists?

Since we have to create the directory in the first place, couldn't we just not link if it already exists?

That doesn't seem great - that won't warn or setup the toolchain if the user has already run x.py build but not created a rustup toolchain.

I think this is why @Mark-Simulacrum suggested using a stamp file. We could also extend this by adding configuration for it in the config.toml; e.g. toolchain-overwrite or something where the user can specify if they want to overwrite the toolchain every time they run x.py setup or not, and by default ./x.py setup will only do it once.

I really don't think this needs additional configuration. We can just check if rustup toolchain list contains stage1.

src/bootstrap/setup.rs

workingjubilee

Have we considered the possible failure mode of a distro-installed Rustup that is later overshadowed by Rustup's normal /home/user/.{cargo,rustup} install process? Possibly that doesn't matter: maybe such a case would make one expect possible overshadowing.

workingjubilee · 2021-09-24T02:21:29Z

src/bootstrap/setup.rs

+        println!("WARNING: `rustup`failed to link stage 1 build to `stage1` toolchain");
+        println!(
+            "To manually link stage 1 build to `stage1` tool chain, run:\n
+            `rustup toolchain link stage1 build/{}/stage1`",
+            build.rustc_target_arg()


So... I could have missed something about the previous code, but at this point in the program, what proof do we have that the user has Rustup on their system that justifies emitting this message all "WARNING!" style, telling them about a binary that possibly isn't installed on their system? It is entirely possible that the user is bootstrapping Rust on their system for the first time, Rustup is not available for this target, and we should be careful to neither alarm nor distract them. Something gentler and softer-voiced seems appropriate.

If rustup is installed:

Linking `stage1` toolchain

If not:

`rustup` is not installed Unable to link `stage1` toolchain

I think it would be fine to only warn if the profile isn't "user" (which is what distros should be using); for everyone else they would find rustup really useful for rust-lang/rust even if they wouldn't use it normally.

It's mostly about framing: gently observing that they could do so is fine, as long as it doesn't appear on every invocation after, if they decide they don't want to use it.

@workingjubilee yup - this is only running once when you first run setup, it's never repeated.

src/bootstrap/setup.rs

workingjubilee · 2021-09-24T02:29:39Z

src/bootstrap/setup.rs

+    let build = TargetSelection::from_user(&env!("BUILD_TRIPLE"));
+    let stage_path = ["build", build.rustc_target_arg(), "stage1"].join("/");
+    let toolchain_link =
+        Command::new("rustup").args(&["toolchain", "link", "stage1", &stage_path[..]]).output();


Usually this command is executed by the user, so like most Unix-y commands, dispatches as silently and swiftly as possible, and alerting someone is a sign of failure.

jyn514 · 2021-09-24T02:50:02Z

Have we considered the possible failure mode of a distro-installed Rustup that is later overshadowed by Rustup's normal /home/user/.{cargo,rustup} install process?

I'm not sure how this could go wrong. If there's a distro-installed rustup, it could run toolchain link just like a binary-installed rustup. toolchain link doesn't change the default, just add additional toolchains.

workingjubilee · 2021-09-24T02:51:13Z

I more meant user surprisal due to the overshadowing and the linked toolchain "disappearing", if that mattered.

jyn514 · 2021-09-24T03:01:06Z

I clarified offline - the way this can go wrong is:

Someone has a version of rustup installed from their distro (in /usr/bin/rustup or wherever).
They build rust from source for reasons and x.py setup runs rustup toolchain link stage1.
They install a new version of rustup so they can download new toolchains.
The linked toolchain disappears.

Given that in this (somewhat convoluted) scenario they'll be able to easily rerun toolchain link or just run rustup install, I'm ok with the toolchain disappearing in that case; I don't see a way to avoid it and the benefit here massively outweighs the risk IMO.

src/bootstrap/setup.rs

jyn514

Looks pretty good to me :) I just have a couple last nits.

@Mark-Simulacrum do you want to take a look before I approve? No problem if not, this seems pretty low risk.

jyn514 · 2021-09-25T14:39:38Z

src/bootstrap/setup.rs

+                } else {
+                    println!(
+                        "Added `stage1` rustup toolchain; try `cargo +stage1 build` on a separate rust project to run a newly-built toolchain"
+                    );
+                }
+            } else {
+                println!(
+                    "`stage1` toolchain already linked; not attempting to link `stage1` toolchain",
+                );
+            }
+        } else {
+            println!(
+                "`rustup` failed to list current toolchains; not attempting to link `stage1` toolchain"
+            );
+        }
+    } else if profile != Profile::User {
+        println!("`rustup` is not installed; cannot link `stage1` toolchain");
+    }


This long list of else blocks makes it hard to tell which error corresponds with which check IMO - could you instead use a separate functions and early returns so it's more clear? Something like this:

maybe_create_rustup_toolchain(); // ... } fn maybe_create_rustup_toolchain() { if !rustup_installed || !stage_dir_exists { // ... return } // ... }

src/bootstrap/setup.rs

Mark-Simulacrum · 2021-09-25T14:50:57Z

No, happy to let you own the review here. Can always adjust later in future PRs if it becomes necessary :)

jyn514 · 2021-09-26T16:05:29Z

@bors r+

This is awesome, thank you! ❤️

bors · 2021-09-26T16:05:30Z

📌 Commit 91f77f7b8850d10b263fc6f35ed1781933c9eb37 has been approved by jyn514

Mark-Simulacrum · 2021-09-26T17:41:03Z

@bors r-

Please squash the commits here to keep history a little cleaner.

Fixed types Add checks for rustup and if toolchain is linked Fortified rustup/directory checks; made other suggested changes Added check for output status Remove output of rustup from console Made suggested change Deleted confusing comment Fixed compiler error; removed extra declaration Refactored to smaller components; made suggested changes Automate toolchain linking for stage 1 builds

jyn514 · 2021-09-28T05:22:42Z

@bors r+ rollup

bors · 2021-09-28T05:22:43Z

📌 Commit adbb608 has been approved by jyn514

x.py: run `rustup toolchain link` in setup Addresses rust-lang#89206 r? `@jyn514`

…laumeGomez Rollup of 8 pull requests Successful merges: - rust-lang#87260 (Libgccjit codegen) - rust-lang#89212 (x.py: run `rustup toolchain link` in setup) - rust-lang#89233 (Hide `<...> defined here` note if the source is not available) - rust-lang#89235 (make junit output more consistent with default format) - rust-lang#89255 (Fix incorrect disambiguation suggestion for associated items) - rust-lang#89276 (Fix the population of the `union.impls` field) - rust-lang#89283 (Add regression test for issue rust-lang#83564) - rust-lang#89318 (rustc_session: Remove lint store from `Session`) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

…etup, r=Mark-Simulacrum Fix linking stage1 toolchain in `./x.py setup` Closes [92319](rust-lang#92319) Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by `@Sl1mb0.` We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497). cc `@jyn514` and `@Mark-Simulacrum` as they were active in rust-lang#89206

…etup, r=Mark-Simulacrum Fix linking stage1 toolchain in `./x.py setup` Closes [92319](rust-lang#92319) Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by ``@Sl1mb0.`` We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497). cc ``@jyn514`` and ``@Mark-Simulacrum`` as they were active in rust-lang#89206

…etup, r=Mark-Simulacrum Fix linking stage1 toolchain in `./x.py setup` Closes [92319](rust-lang#92319) Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by ```@Sl1mb0.``` We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497). cc ```@jyn514``` and ```@Mark-Simulacrum``` as they were active in rust-lang#89206

…etup, r=Mark-Simulacrum Fix linking stage1 toolchain in `./x.py setup` Closes [92319](rust-lang#92319) Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by ````@Sl1mb0.```` We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497). cc ````@jyn514```` and ````@Mark-Simulacrum```` as they were active in rust-lang#89206

…etup, r=Mark-Simulacrum Fix linking stage1 toolchain in `./x.py setup` Closes [92319](rust-lang#92319) Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by `````@Sl1mb0.````` We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497). cc `````@jyn514````` and `````@Mark-Simulacrum````` as they were active in rust-lang#89206

rust-highfive assigned jyn514 Sep 24, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 24, 2021

jyn514 suggested changes Sep 24, 2021

View reviewed changes

jyn514 changed the title ~~Xpy toolchain link~~ x.py: run rustup toolchain link in setup Sep 24, 2021

workingjubilee reviewed Sep 24, 2021

View reviewed changes

jyn514 approved these changes Sep 24, 2021

View reviewed changes

src/bootstrap/setup.rs Outdated Show resolved Hide resolved

src/bootstrap/setup.rs Outdated Show resolved Hide resolved

src/bootstrap/setup.rs Outdated Show resolved Hide resolved

src/bootstrap/setup.rs Outdated Show resolved Hide resolved

jyn514 reviewed Sep 24, 2021

View reviewed changes

src/bootstrap/setup.rs Outdated Show resolved Hide resolved

jyn514 reviewed Sep 24, 2021

View reviewed changes

src/bootstrap/setup.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

jyn514 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 24, 2021

jyn514 reviewed Sep 25, 2021

View reviewed changes

jyn514 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 25, 2021

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 26, 2021

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 26, 2021

Sl1mb0 force-pushed the xpy-toolchain-link branch from 91f77f7 to adbb608 Compare September 27, 2021 04:27

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 28, 2021

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request Sep 28, 2021

Rollup merge of rust-lang#89212 - Sl1mb0:xpy-toolchain-link, r=jyn514

6edb5c0

x.py: run `rustup toolchain link` in setup Addresses rust-lang#89206 r? `@jyn514`

This was referenced Sep 28, 2021

Rollup of 9 pull requests #89330

Closed

Rollup of 8 pull requests #89331

Merged

bors merged commit 91da29f into rust-lang:master Sep 29, 2021

rustbot added this to the 1.57.0 milestone Sep 29, 2021

Sl1mb0 deleted the xpy-toolchain-link branch October 5, 2021 17:17

jyn514 mentioned this pull request Dec 27, 2021

x.py setup stopped being able to link toolchains at some point #92319

Closed

yerke mentioned this pull request Jan 30, 2022

Fix linking stage1 toolchain in ./x.py setup #93487

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x.py: run `rustup toolchain link` in setup #89212

x.py: run `rustup toolchain link` in setup #89212

Sl1mb0 commented Sep 24, 2021 •

edited

Loading

jyn514 Sep 24, 2021

workingjubilee Sep 24, 2021

jyn514 Sep 24, 2021

workingjubilee Sep 24, 2021

Sl1mb0 Sep 24, 2021

jyn514 Sep 24, 2021

Sl1mb0 Sep 24, 2021

jyn514 Sep 24, 2021

workingjubilee left a comment

workingjubilee Sep 24, 2021

Sl1mb0 Sep 24, 2021 •

edited

Loading

jyn514 Sep 24, 2021

workingjubilee Sep 24, 2021 •

edited

Loading

jyn514 Sep 24, 2021

workingjubilee Sep 24, 2021

jyn514 commented Sep 24, 2021

workingjubilee commented Sep 24, 2021

jyn514 commented Sep 24, 2021 •

edited

Loading

This comment has been minimized.

jyn514 left a comment

jyn514 Sep 25, 2021

Mark-Simulacrum commented Sep 25, 2021

jyn514 commented Sep 26, 2021

bors commented Sep 26, 2021

Mark-Simulacrum commented Sep 26, 2021

jyn514 commented Sep 28, 2021

bors commented Sep 28, 2021

x.py: run rustup toolchain link in setup #89212

x.py: run rustup toolchain link in setup #89212

Conversation

Sl1mb0 commented Sep 24, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

workingjubilee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Sl1mb0 Sep 24, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

workingjubilee Sep 24, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jyn514 commented Sep 24, 2021

workingjubilee commented Sep 24, 2021

jyn514 commented Sep 24, 2021 • edited Loading

This comment has been minimized.

jyn514 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mark-Simulacrum commented Sep 25, 2021

jyn514 commented Sep 26, 2021

bors commented Sep 26, 2021

Mark-Simulacrum commented Sep 26, 2021

jyn514 commented Sep 28, 2021

bors commented Sep 28, 2021

x.py: run `rustup toolchain link` in setup #89212

x.py: run `rustup toolchain link` in setup #89212

Sl1mb0 commented Sep 24, 2021 •

edited

Loading

Sl1mb0 Sep 24, 2021 •

edited

Loading

workingjubilee Sep 24, 2021 •

edited

Loading

jyn514 commented Sep 24, 2021 •

edited

Loading