rust-src component installed by `rustup component add` not seen by `rustup toolchain install ... --component`. #2601

eddyb · 2020-12-08T13:56:09Z

I would expect rustup toolchain install ... --component rust-src to do nothing when rust-src is already installed.
But if it installed through rustup component add rust-src, it tries installing again (and fails due to conflicting with itself):

setup

$ rustup toolchain remove nightly
info: uninstalling toolchain 'nightly-x86_64-unknown-linux-gnu'
info: toolchain 'nightly-x86_64-unknown-linux-gnu' uninstalled
$ rustup show profile
default
$ rustup toolchain install nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2020-12-08, rust version 1.50.0-nightly (3d6705aa5 2020-12-07)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'rust-docs'
 14.4 MiB /  14.4 MiB (100 %)   8.0 MiB/s in  1s ETA:  0s
info: installing component 'rust-std'
 24.6 MiB /  24.6 MiB (100 %)   9.7 MiB/s in  2s ETA:  0s
info: installing component 'rustc'
 56.1 MiB /  56.1 MiB (100 %)  11.1 MiB/s in  5s ETA:  0s
info: installing component 'rustfmt'

  nightly-x86_64-unknown-linux-gnu installed - rustc 1.50.0-nightly (3d6705aa5 2020-12-07)

$ rustup component add --toolchain nightly rust-src
info: downloading component 'rust-src'
info: installing component 'rust-src'

failure

$ rustup toolchain install nightly --component rust-src
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2020-12-08, rust version 1.50.0-nightly (3d6705aa5 2020-12-07)
info: downloading component 'rust-src'
info: installing component 'rust-src'
info: rolling back changes
error: failed to install component: 'rust-src-x86_64-unknown-linux-gnu', detected conflict: '"lib/rustlib/src/rust/Cargo.lock"'

expectation (AFAICT, this is what happens for any component other than rust-src)

$ rustup toolchain install nightly --component rust-std
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2020-12-08, rust version 1.50.0-nightly (3d6705aa5 2020-12-07)
info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date

  nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.50.0-nightly (3d6705aa5 2020-12-07)

it even works correctly for other components that aren't part of the original installation either:

$ rustup component add --toolchain nightly rustc-dev
info: downloading component 'rustc-dev'
 93.7 MiB /  93.7 MiB (100 %)  26.9 MiB/s in  3s ETA:  0s
info: installing component 'rustc-dev'
 93.7 MiB /  93.7 MiB (100 %)  10.8 MiB/s in  8s ETA:  0s
$ rustup toolchain install nightly --component rustc-dev
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2020-12-08, rust version 1.50.0-nightly (3d6705aa5 2020-12-07)
info: component 'rustc-dev' for target 'x86_64-unknown-linux-gnu' is up to date

  nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.50.0-nightly (3d6705aa5 2020-12-07)

We've hit this in the wild because the Rust (RLS) VSCode extension uses rustup component add ... to install components, whereas in Rust-GPU we use rustup toolchain install ... --component ... to install components.

Attempting to reproduce this was really difficult without doing a bunch of reinstalls from scratch, it seems that rustup auto-corrects for some incorrect state on its own sometimes, for example I got a very weird result from running this twice:

rustup component add --toolchain nightly rust-std

Which I was able to reproduce on top of the above "setup", like this:

$ rustup component remove --toolchain nightly rust-src
info: removing component 'rust-src'
$ rustup toolchain install nightly --component rust-src
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2020-12-08, rust version 1.50.0-nightly (3d6705aa5 2020-12-07)
info: downloading component 'rust-src'
info: installing component 'rust-src'

  nightly-x86_64-unknown-linux-gnu updated - rustc 1.50.0-nightly (3d6705aa5 2020-12-07) (from rustc 1.50.0-nightly (3d6705aa5 2020-12-07))

$ rustup component add --toolchain nightly rust-std
info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date
info: downloading component 'rust-src'
info: removing component 'rust-src'
info: installing component 'rust-src'
$ rustup component add --toolchain nightly rust-std
info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date

EDIT: forgot to mention that the version is rustup 1.21.1

The text was updated successfully, but these errors were encountered:

kinnison · 2020-12-08T14:05:23Z

Okay that's an interesting set of behaviours for me to puzzle over

eddyb · 2020-12-08T14:07:53Z

Okay found the cause, ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/components contains:

rust-src when installed via rustup component add --toolchain nightly rust-src
rust-src-x86_64-unknown-linux-gnu when installed via rustup toolchain install nightly --component rust-src

eddyb · 2020-12-08T14:09:07Z

~~I'm guessing when rust-src-x86_64-unknown-linux-gnu is installed, trying to install rust-src recognizes it as "already installed", whereas the other way around, it doesn't recognize it.~~

EDIT: it's actually broken both ways, I just wasn't testing enough sequences of commands, see #2601 (comment).

kinnison · 2020-12-08T14:15:15Z

Yes, there's definitely something up with how we're resolving the architecture there. rust-src is unusual in that it's an all-arch component whereas the others are per-arch.

eddyb · 2020-12-09T07:35:59Z

Looks like the other way around is broken too, sorry!

$ rustup component remove --toolchain nightly rust-src
info: removing component 'rust-src'
$ rustup toolchain install nightly --component rust-src
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2020-12-09, rust version 1.50.0-nightly (1700ca07c 2020-12-08)
info: downloading component 'rust-src'
info: installing component 'rust-src'

  nightly-x86_64-unknown-linux-gnu updated - rustc 1.50.0-nightly (1700ca07c 2020-12-08) (from rustc 1.50.0-nightly (1700ca07c 2020-12-08))

$ rustup component add --toolchain nightly rust-src
info: downloading component 'rust-src'
info: downloading component 'rust-src'
info: removing component 'rust-src'
info: installing component 'rust-src'
info: installing component 'rust-src'
info: rolling back changes
error: could not rename component file from '/home/eddy/.rustup/tmp/9u3gxcem2es8sx16_dir/bk' to '/home/eddy/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src'
error: could not rename component file from '/home/eddy/.rustup/tmp/zc8eo__o5fcn9_ob_dir/bk' to '/home/eddy/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust'
error: could not rename component file from '/home/eddy/.rustup/tmp/55kylnxpm88kdx2h_dir/bk' to '/home/eddy/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library'
error: could not rename component file from '/home/eddy/.rustup/tmp/ejy16mvpbilnbe6b_dir/bk' to '/home/eddy/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/stdarch'
...
error: could not rename component file from '/home/eddy/.rustup/tmp/21mib900f87c1u92_dir/bk' to '/home/eddy/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/profiler_builtins'
error: could not rename component file from '/home/eddy/.rustup/tmp/m6_5dgv9_ung340j_dir/bk' to '/home/eddy/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/vendor/rustc-std-workspace-core'
error: could not rename component file from '/home/eddy/.rustup/tmp/slzyjt969y1rk1wq_dir/bk' to '/home/eddy/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/term/src/terminfo/parser'
error: failed to install component: 'rust-src', detected conflict: '"lib/rustlib/src/rust/Cargo.lock"'

(it lists with every single one of the 152 directories/files in the component, spamming my terminal)

For whatever reason, specifying rust-std as well does make it work (even in the case above where rust-src was installed through rustup toolchain install ... --components rust-src), so I'm going to use this as the workaround:

$ rustup component add --toolchain nightly rust-std rust-src
info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date
info: downloading component 'rust-src'
info: removing component 'rust-src'
info: installing component 'rust-src'
info: component 'rust-src' is up to date

And the resulting lib/rustlib/component files contains rust-src, which is the desired outcome IMO.

eddyb · 2020-12-09T11:23:12Z

Ah, looking at at the CHANGELOG entry for 1.21.0, I'm guessing #2087 and/or #2115 didn't go far enough?

EDIT: wait, I'm on rustup 1.21.1, will need to check on latest rustup later today, in case this was fixed since.

kinnison · 2020-12-09T11:47:42Z

It'd be worth double-checking, but I don't think I did much in the way of altering the deduplication inputs to toolchain installation. When we attempt to "install" an already installed toolchain we have to look at the requested components and deduplicate against the installed components. I wonder if we're failing to do that correctly for rust-src.

eddyb · 2020-12-09T12:27:42Z

Testing with below rust-toolchain on rustup 1.23.1, it seems to behave like rustup toolchain install ... --components rust-src (which makes a lot of sense), including the bug with running rustup component add rust-src later, described in #2601 (comment).

[toolchain]
channel = "nightly"
components = [ "rust-src" ]

eddyb · 2020-12-09T13:05:29Z

Heh, RLS (or rather, the VSCode extension) plays well with the TOML rust-toolchain we need, because of a coincidence:

> Executing task: rustup component add rust-analysis --toolchain nightly-2020-11-24-x86_64-unknown-linux-gnu <

info: downloading component 'rust-analysis'
info: downloading component 'rust-src'
info: removing component 'rust-src'
info: installing component 'rust-analysis'
info: using up to 500.0 MiB of RAM to unpack components
info: installing component 'rust-src'

Terminal will be reused by tasks, press any key to close it.

> Executing task: rustup component add rust-src --toolchain nightly-2020-11-24-x86_64-unknown-linux-gnu <

info: component 'rust-src' is up to date

That is, rust-analysis behaves like rust-std, in that it forces rust-src to fix itself - if rustup component add rust-src came first, it would've failed.

kinnison · 2020-12-09T14:05:41Z

Thank you for the updates. I intend to have a go at bottoming this out this week.

Sometimes someone might choose an additional component to add during an install/update which is wildcarded. We need to ensure that we detect this and request the install with the wildcard otherwise we can hit situations such as that identified in rust-lang#2601 Signed-off-by: Daniel Silverstone <[email protected]>

eddyb added the bug label Dec 8, 2020

kinnison self-assigned this Dec 8, 2020

kinnison added this to the 1.24.0 milestone Dec 8, 2020

eddyb mentioned this issue Dec 9, 2020

Simplify setup.{sh,bat} and work around rust-lang/rustup#2601. EmbarkStudios/rust-gpu#332

Closed

eddyb mentioned this issue Dec 9, 2020

Update to use rust-toolchain toml EmbarkStudios/rust-gpu#284

Merged

kinnison mentioned this issue Dec 9, 2020

Fix 2601 #2602

Merged

kinnison closed this as completed in #2602 Dec 24, 2020

rami3l mentioned this issue Nov 12, 2023

Flaky conflict installing rust-src on GitHub actions runners #3530

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rust-src component installed by `rustup component add` not seen by `rustup toolchain install ... --component`. #2601

rust-src component installed by `rustup component add` not seen by `rustup toolchain install ... --component`. #2601

eddyb commented Dec 8, 2020 •

edited

Loading

kinnison commented Dec 8, 2020

eddyb commented Dec 8, 2020

eddyb commented Dec 8, 2020 •

edited

Loading

kinnison commented Dec 8, 2020

eddyb commented Dec 9, 2020 •

edited

Loading

eddyb commented Dec 9, 2020 •

edited

Loading

kinnison commented Dec 9, 2020

eddyb commented Dec 9, 2020 •

edited

Loading

eddyb commented Dec 9, 2020

kinnison commented Dec 9, 2020

rust-src component installed by rustup component add not seen by rustup toolchain install ... --component. #2601

rust-src component installed by rustup component add not seen by rustup toolchain install ... --component. #2601

Comments

eddyb commented Dec 8, 2020 • edited Loading

kinnison commented Dec 8, 2020

eddyb commented Dec 8, 2020

eddyb commented Dec 8, 2020 • edited Loading

kinnison commented Dec 8, 2020

eddyb commented Dec 9, 2020 • edited Loading

eddyb commented Dec 9, 2020 • edited Loading

kinnison commented Dec 9, 2020

eddyb commented Dec 9, 2020 • edited Loading

eddyb commented Dec 9, 2020

kinnison commented Dec 9, 2020

rust-src component installed by `rustup component add` not seen by `rustup toolchain install ... --component`. #2601

rust-src component installed by `rustup component add` not seen by `rustup toolchain install ... --component`. #2601

eddyb commented Dec 8, 2020 •

edited

Loading

eddyb commented Dec 8, 2020 •

edited

Loading

eddyb commented Dec 9, 2020 •

edited

Loading

eddyb commented Dec 9, 2020 •

edited

Loading

eddyb commented Dec 9, 2020 •

edited

Loading