don't download std-docs on CI #7513

Eh2406 · 2019-10-15T15:19:08Z

Now that we have a way to do it, lets not download std-docs on CI. Based on the blog post:
https://blog.rust-lang.org/2019/10/15/Rustup-1.20.0.html

rust-highfive · 2019-10-15T15:19:11Z

r? @ehuss

(rust_highfive has picked a reviewer for you, use r? to override)

Eh2406 · 2019-10-15T15:54:25Z

Windows strikes again. I think: rustup self update has changed the exe that rustup points to. So windows won't let us use rustup until we reopen powershell. Not sure the cleanest way to proceed.

alexcrichton · 2019-10-16T14:26:21Z

It looks like it works now though, right? The changes here look good to me!

ehuss · 2019-10-16T14:36:59Z

This looks good to me, too.

However, the performance improvement is a little disappointing. The Windows builder went from 2m 33s to 2m 7s. The linux builders have very little impact (dropping from about 28s to 25s). I don't have any ideas why the Windows is still so slow (does maybe deleting the component take a long time?). Anyway, I'd be fine to merge this.

tesuji

The curl command works on all OSes.
See example here: https://github.com/lzutao/home/runs/261056912#step:3:1

tesuji · 2019-10-17T20:47:03Z

ci/azure-install-rust.yml

        echo "##vso[task.prependpath]$HOME/.cargo/bin"
      fi
    displayName: Install rustup

  - bash: |
      set -e
+      rustup set profile minimal
+      rustup component remove --toolchain=$TOOLCHAIN rust-docs || echo "already removed"
      rustup update $TOOLCHAIN


Suggested change

rustup update $TOOLCHAIN

rustup update --no-self-update $TOOLCHAIN

tesuji · 2019-10-17T20:47:04Z

ci/azure-install-rust.yml

@@ -6,16 +6,18 @@ steps:
        rustup self update
      elif [ "$AGENT_OS" = "Windows_NT" ]; then
        curl -sSf -o rustup-init.exe https://win.rustup.rs
-        rustup-init.exe -y --default-toolchain $TOOLCHAIN
+        rustup-init.exe -y --default-toolchain $TOOLCHAIN --profile=minimal


This works on all OSes

Suggested change

rustup-init.exe -y --default-toolchain $TOOLCHAIN --profile=minimal

curl -sSL https://sh.rustup.rs | sh -s -- -y --default-toolchain=$TOOLCHAIN --profile=minimal

Sorry for the delay. I am now back from vacation. How do we best combine the branches?

How about using a custom script: https://github.com/lzutao/junction/blob/ci-scripts/ci/install-rustup.bash ?

ehuss · 2019-10-17T21:08:50Z

Hm, it really does take 70s to delete the rust-docs component. Well, hopefully some day in the future the default install will use the minimal profile so it won't be necessary.

alexcrichton · 2019-10-18T13:45:39Z

That may be a bug in rustup perhaps that it takes so long to delete docs? I suspect that actually deleting the component shouldn't take 70 seconds

ehuss · 2019-10-23T17:14:14Z

I don't think it is an issue with rustup. I see similar times on other CI systems. On my very fast local system with a very fast SSD, it takes about 13s to delete them. Doing the same thing inside a docker container on the same system takes over 35s. Using rm -r takes the same amount of time as rustup remove. It is not too surprising that the virtual images on azure are a little slower. I think Windows file IO is just really slow.

alexcrichton · 2019-11-04T14:57:33Z

r=me on the changes whenever this is ready, but unfortunately I think our nightly CI has regressed

ehuss · 2019-11-04T16:15:45Z

Yea, we need to add the rustc-dev component in rustup, due to rust-lang/rust#65474.

ehuss · 2019-11-04T20:22:28Z

ci/azure-install-rust.yml

-      rustup update $TOOLCHAIN
+      rustup set profile minimal
+      rustup component remove --toolchain=$TOOLCHAIN rust-docs || echo "already removed"
+      rustup component add --toolchain=$TOOLCHAIN rustc-dev || echo "already installed"


Maybe move it after the call to rustup update? I think it will fail otherwise, because nightly is not already installed, and "component add" won't be remembered. Also, the echo "already installed" might be a little misleading, since it gobbles any error (such as doesn't exist, or download failure).

Also, maybe it should only install rustc-dev when using the nightly toolchain? It won't be used for any of the others. I'm also not sure if will be published for the stable toolchains?

Ok. Switched the order and removed the echo. Now everything brakes. How can we install it only on nightly, I is not so experienced with azure?

Oh, I was just thinking adding a conditional to the bash script, maybe something like:

if [ "$TOOLCHAIN" = "nightly" ]; then rustup component add --toolchain=$TOOLCHAIN rustc-dev fi

just after rustup update.

You can also add it as a separate step with a condition, but I don't think it is necessary.

Eh2406 · 2019-11-04T21:59:18Z

CI is green!

ehuss · 2019-11-04T22:58:36Z

Awesome, thanks!

@bors r+

bors · 2019-11-04T22:58:38Z

📌 Commit e028b87 has been approved by ehuss

bors · 2019-11-04T22:58:44Z

⌛ Testing commit e028b87 with merge ba5cca2...

don't download std-docs on CI Now that we have a way to do it, lets not download std-docs on CI. Based on the blog post: https://blog.rust-lang.org/2019/10/15/Rustup-1.20.0.html

bors · 2019-11-04T23:21:50Z

☀️ Test successful - checks-azure
Approved by: ehuss
Pushing ba5cca2 to master...

Update Cargo, books ## cargo 12 commits in 5da4b4d47963868d9878480197581ccbbdaece74..8280633db680dec5bfe1de25156d1a1d53e6d190 2019-10-28 21:53:41 +0000 to 2019-11-11 23:17:05 +0000 - Don't panic when parsing `/proc/stat` (rust-lang/cargo#7580) - Fix unused configuration key warning for a few keys under `build`. (rust-lang/cargo#7575) - Add back support for `BROWSER` envvar in `cargo doc --open`. (rust-lang/cargo#7576) - Only include "already existing ..." comment in gitignore on conflict (rust-lang/cargo#7570) - Add VS Code user dir to .gitignore (rust-lang/cargo#7578) - Added aliases to subcommand typo suggestions. (rust-lang/cargo#7486) - Use multiple requirement syntax consistently (rust-lang/cargo#7573) - Update verison to 0.42 (rust-lang/cargo#7568) - Expand documentation on build scripts. (rust-lang/cargo#7565) - Update crossbeam-utils requirement from 0.6 to 0.7 (rust-lang/cargo#7566) - don't download std-docs on CI (rust-lang/cargo#7513) - Change my-buddy to github-handle (rust-lang/cargo#7553) ## nomicon 2 commits in 5004ad30d69f93553ceef74439fea2159d1f769e..58e36e0e08dec5a379ac568827c058e25990d6cd 2019-10-12 19:52:40 +0200 to 2019-10-30 08:14:24 -0500 - remove references to the nursery - Add github action to replace Travis.yml (rust-lang/nomicon#172) ## reference 7 commits in 4b21b64..45558c4 2019-10-27 22:33:11 +0100 to 2019-11-08 14:47:35 +0100 - Audit code blocks. (rust-lang/reference#715) - Update coherence and orphan rules documentation to match RFC 2451 (rust-lang/reference#703) - Update organization name (rust-lang/reference#713) - State that no_implicit_prelude also applies to nested modules (rust-lang/reference#707) - expand Copy docs (rust-lang/reference#711) - github action doesn't use the nursery (rust-lang/reference#706) - Migrate to GitHub Actions. (rust-lang/reference#705) ## book 1 commits in 28fa3d15b0bc67ea5e79eeff2198e4277fc61baf..e79dd62aa63396714278d484d91d48826737f47f 2019-10-29 07:16:09 -0500 to 2019-10-30 07:33:12 -0500 - No need for an iterator here to fetch values (rust-lang/book#1957) ## rust-by-example 1 commits in f3197ddf2abab9abdbc029def8164f4a748b0d91..dcee312c66267eb5a2f6f1561354003950e29105 2019-10-29 10:17:40 -0300 to 2019-10-31 11:26:53 -0300 - refactor: simplify extracting Result from Option (rust-lang/rust-by-example#1283) ## edition-guide 2 commits in e58bc4ca104e890ac56af846877c874c432a64b5..f553fb26c60c4623ea88a1cfe731eafe0643ce34 2019-07-31 20:14:12 +0200 to 2019-10-30 08:27:42 -0500 - remove old references to the nursery - Port from Travis to GitHub Actions (rust-lang/edition-guide#192)

rust-highfive assigned ehuss Oct 15, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 15, 2019

Eh2406 closed this Oct 15, 2019

Eh2406 reopened this Oct 15, 2019

tesuji reviewed Oct 17, 2019

View reviewed changes

Eh2406 added 4 commits November 2, 2019 21:33

dont download std-docs on CI

5b658ab

widows is annoying

b5c5900

update is annoying

d1de600

use no-self-update

397dd68

Eh2406 force-pushed the ci branch from 3e367d6 to 397dd68 Compare November 3, 2019 01:40

use curl on windows

e4d5563

ehuss reviewed Nov 4, 2019

View reviewed changes

add rustc-dev

5d35d90

Eh2406 force-pushed the ci branch from 7127e55 to 5d35d90 Compare November 4, 2019 20:31

only on nightly

e028b87

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 4, 2019

bors added a commit that referenced this pull request Nov 4, 2019

Auto merge of #7513 - Eh2406:ci, r=ehuss

ba5cca2

don't download std-docs on CI Now that we have a way to do it, lets not download std-docs on CI. Based on the blog post: https://blog.rust-lang.org/2019/10/15/Rustup-1.20.0.html

bors merged commit e028b87 into rust-lang:master Nov 4, 2019

ehuss mentioned this pull request Nov 12, 2019

Update Cargo, books rust-lang/rust#66336

Merged

ehuss added this to the 1.41.0 milestone Feb 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

don't download std-docs on CI #7513

don't download std-docs on CI #7513

Eh2406 commented Oct 15, 2019

rust-highfive commented Oct 15, 2019

Eh2406 commented Oct 15, 2019

alexcrichton commented Oct 16, 2019

ehuss commented Oct 16, 2019

tesuji left a comment •

edited

Loading

tesuji Oct 17, 2019 •

edited

Loading

tesuji Oct 17, 2019

Eh2406 Nov 3, 2019

tesuji Nov 3, 2019

ehuss commented Oct 17, 2019

alexcrichton commented Oct 18, 2019

ehuss commented Oct 23, 2019

alexcrichton commented Nov 4, 2019

ehuss commented Nov 4, 2019

ehuss Nov 4, 2019

Eh2406 Nov 4, 2019

ehuss Nov 4, 2019

Eh2406 commented Nov 4, 2019

ehuss commented Nov 4, 2019

bors commented Nov 4, 2019

bors commented Nov 4, 2019

bors commented Nov 4, 2019

	rustup update $TOOLCHAIN
	rustup update --no-self-update $TOOLCHAIN

	rustup-init.exe -y --default-toolchain $TOOLCHAIN --profile=minimal
	curl -sSL https://sh.rustup.rs \| sh -s -- -y --default-toolchain=$TOOLCHAIN --profile=minimal

don't download std-docs on CI #7513

don't download std-docs on CI #7513

Conversation

Eh2406 commented Oct 15, 2019

rust-highfive commented Oct 15, 2019

Eh2406 commented Oct 15, 2019

alexcrichton commented Oct 16, 2019

ehuss commented Oct 16, 2019

tesuji left a comment • edited Loading

Choose a reason for hiding this comment

tesuji Oct 17, 2019 • edited Loading

Choose a reason for hiding this comment

tesuji Oct 17, 2019

Choose a reason for hiding this comment

Eh2406 Nov 3, 2019

Choose a reason for hiding this comment

tesuji Nov 3, 2019

Choose a reason for hiding this comment

ehuss commented Oct 17, 2019

alexcrichton commented Oct 18, 2019

ehuss commented Oct 23, 2019

alexcrichton commented Nov 4, 2019

ehuss commented Nov 4, 2019

ehuss Nov 4, 2019

Choose a reason for hiding this comment

Eh2406 Nov 4, 2019

Choose a reason for hiding this comment

ehuss Nov 4, 2019

Choose a reason for hiding this comment

Eh2406 commented Nov 4, 2019

ehuss commented Nov 4, 2019

bors commented Nov 4, 2019

bors commented Nov 4, 2019

bors commented Nov 4, 2019

tesuji left a comment •

edited

Loading

tesuji Oct 17, 2019 •

edited

Loading