azure: Convert Windows installations scripts to `bash` #64553

alexcrichton · 2019-09-17T16:01:05Z

Looks like script, which uses cmd.exe, doesn't have fail-fast
behavior and if a leading command fails the script doesn't actually fail
so long as the last command succeeds. We instead want the opposite
behavior where if any step fails the whole script fails.

I don't really know cmd.exe that well, nor powershell, so I've opted
to move everything to bash which should be a good common denominator
amongst all platforms to work with. Additionally I know that set -e
works to cause scripts to fail fast.

Closes #64551

rust-highfive · 2019-09-17T16:01:19Z

r? @Mark-Simulacrum

(rust_highfive has picked a reviewer for you, use r? to override)

Mark-Simulacrum · 2019-09-17T16:12:39Z

src/ci/azure-pipelines/steps/install-windows-build-deps.yml

    echo ##vso[task.prependpath]C:\Program Files (x86)\Inno Setup 5
+    curl.exe -o is-install.exe https://rust-lang-ci-mirrors.s3-us-west-1.amazonaws.com/rustc/2017-08-22-is.exe
+    is-install.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART /SP-


Hm, Is there a reason we're moving this after the echo statement? It looks like these two might be connected

Ah yeah I couldn't figure out how to get this working with bash and AFAIK the exit code of the script is the last statement, so the thing which may fail is listed here last and the echo, which always succeeds, was moved above it.

Mark-Simulacrum · 2019-09-17T16:19:10Z

@bors r+

bors · 2019-09-17T16:19:11Z

📌 Commit eb6833303f37a190100447a8bb9fd558e91eb31d has been approved by Mark-Simulacrum

Mark-Simulacrum · 2019-09-17T16:26:15Z

Oh, actually, I forgot I left that comment towards the start of my review :)

r=me with an answer/fix

@bors r-

alexcrichton · 2019-09-17T17:33:15Z

@bors: r=Mark-Simulacrum

bors · 2019-09-17T17:33:16Z

📌 Commit eb6833303f37a190100447a8bb9fd558e91eb31d has been approved by Mark-Simulacrum

bors · 2019-09-18T10:29:27Z

⌛ Testing commit eb6833303f37a190100447a8bb9fd558e91eb31d with merge 281b666571a153ef0d2dce6688ecc6a16c83e47a...

rust-highfive · 2019-09-18T10:36:31Z

Your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-09-18T10:31:06.8081417Z do so (now or later) by using -b with the checkout command again. Example:
2019-09-18T10:31:06.8081869Z 
2019-09-18T10:31:06.8082359Z   git checkout -b <new-branch-name>
2019-09-18T10:31:06.8083183Z 
2019-09-18T10:31:06.8083698Z HEAD is now at 281b66657 Auto merge of #64553 - alexcrichton:windows-bash-install-scripts, r=Mark-Simulacrum
2019-09-18T10:31:06.8492295Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-09-18T10:31:06.8615551Z ==============================================================================
2019-09-18T10:31:06.8615646Z Task         : Bash
2019-09-18T10:31:06.8615736Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-09-18T10:32:14.4350151Z By installing you accept licenses for the packages.
2019-09-18T10:32:20.2039778Z 
2019-09-18T10:32:20.2040523Z msys2 v20180531.0.0 [Approved]
2019-09-18T10:32:20.2159241Z msys2 package files install completed. Performing other installation steps.
2019-09-18T10:32:23.7535253Z Installing to: D:\a/msys2
2019-09-18T10:32:24.4707124Z Extracting 64-bit C:\ProgramData\chocolatey\lib\msys2\tools\msys2-base-x86_64-20180531.tar.xz to D:\a/msys2...
2019-09-18T10:32:31.8483362Z D:\a/msys2
2019-09-18T10:32:31.8553019Z Extracting D:\a\msys2\msys2-base-x86_64-20180531.tar to D:\a/msys2...
2019-09-18T10:32:52.5582295Z D:\a/msys2
2019-09-18T10:33:01.1552236Z Repeating system update until there are no more updates or max 5 iterations
2019-09-18T10:33:01.1566856Z 
2019-09-18T10:33:01.1567010Z ================= SYSTEM UPDATE 1 =================
2019-09-18T10:33:01.1567061Z 
---
2019-09-18T10:34:09.1671480Z   Software installed to 'D:\a/msys2'
2019-09-18T10:34:09.2171915Z 
2019-09-18T10:34:09.2172382Z Chocolatey installed 1/1 packages. 
2019-09-18T10:34:09.2172623Z  See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
2019-09-18T10:34:19.7969957Z The STDIO streams did not close within 10 seconds of the exit event from process 'C:\Program Files\Git\bin\bash.exe'. This may indicate a child process inherited the STDIO streams and has not yet exited.
2019-09-18T10:34:24.8190055Z ##[section]Starting: Install msys2 base deps
2019-09-18T10:34:24.8321564Z ==============================================================================
2019-09-18T10:34:24.8321662Z Task         : Bash
2019-09-18T10:34:24.8321761Z Description  : Run a Bash script on macOS, Linux, or Windows
2019-09-18T10:34:24.8321761Z Description  : Run a Bash script on macOS, Linux, or Windows
2019-09-18T10:34:24.8321990Z Version      : 3.151.3
2019-09-18T10:34:24.8322080Z Author       : Microsoft Corporation
2019-09-18T10:34:24.8322158Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
2019-09-18T10:34:24.8322269Z ==============================================================================
2019-09-18T10:34:25.0894482Z Generating script.
2019-09-18T10:34:25.0922231Z Script contents:
2019-09-18T10:34:25.0934410Z pacman -S --noconfirm --needed base-devel ca-certificates make diffutils tar
2019-09-18T10:34:25.0998699Z [command]D:\a\msys2\usr\bin\bash.exe --noprofile --norc -c pwd
2019-09-18T10:34:25.1363327Z ========================== Starting Command Output ===========================
2019-09-18T10:34:25.1381857Z [command]D:\a\msys2\usr\bin\bash.exe --noprofile --norc /d/a/_temp/49f55fd4-a66e-4718-9053-5b809e06d8fb.sh
2019-09-18T10:34:25.8359222Z :: There are 56 members in group base-devel:
2019-09-18T10:34:25.8359364Z :: Repository msys
---
2019-09-18T10:35:35.2886945Z 
2019-09-18T10:35:35.2887004Z Scanning the drive for archives:
2019-09-18T10:35:35.2887089Z 1 file, 188813 bytes (185 KiB)
2019-09-18T10:35:35.2887130Z 
2019-09-18T10:35:35.2887199Z Extracting archive: ninja.zip
2019-09-18T10:35:35.2934170Z --
2019-09-18T10:35:35.2934262Z Path = ninja.zip
2019-09-18T10:35:35.2934428Z Physical Size = 188813
2019-09-18T10:35:35.2934579Z 
2019-09-18T10:35:35.2934640Z Everything is Ok
2019-09-18T10:35:35.2934676Z 
---
2019-09-18T10:35:35.6208355Z git config --replace-all --global core.autocrlf false
2019-09-18T10:35:35.6270137Z [command]D:\a\msys2\usr\bin\bash.exe --noprofile --norc -c pwd
2019-09-18T10:35:35.6590739Z /d/a/_temp
2019-09-18T10:35:35.6660398Z ========================== Starting Command Output ===========================
2019-09-18T10:35:35.6681234Z [command]D:\a\msys2\usr\bin\bash.exe --noprofile --norc /d/a/_temp/3aafc9d7-2ec0-42d6-a9b0-b7d3e880780d.sh
2019-09-18T10:35:35.7327172Z error: could not lock config file D:/a/msys2/home/VssAdministrator/.gitconfig: No such file or directory
2019-09-18T10:35:35.7480438Z ##[error]Bash exited with code '255'.
2019-09-18T10:35:35.7590085Z ##[section]Starting: Upload CPU usage statistics
2019-09-18T10:35:35.7704986Z ==============================================================================
2019-09-18T10:35:35.7705096Z Task         : Bash
2019-09-18T10:35:35.7705168Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-09-18T10:35:36.0379485Z aws s3 cp --acl public-read cpu-usage.csv s3://$DEPLOY_BUCKET/rustc-builds/$BUILD_SOURCEVERSION/cpu-$CI_JOB_NAME.csv
2019-09-18T10:35:36.0443355Z [command]D:\a\msys2\usr\bin\bash.exe --noprofile --norc -c pwd
2019-09-18T10:35:36.0777509Z /d/a/_temp
2019-09-18T10:35:36.0827212Z ========================== Starting Command Output ===========================
2019-09-18T10:35:36.0846615Z [command]D:\a\msys2\usr\bin\bash.exe --noprofile --norc /d/a/_temp/987d4b7e-2917-4dff-981c-6a0d555f1618.sh
2019-09-18T10:35:36.1238580Z /d/a/_temp/987d4b7e-2917-4dff-981c-6a0d555f1618.sh: line 1: aws: command not found
2019-09-18T10:35:36.1282856Z ##[error]Bash exited with code '127'.
2019-09-18T10:35:36.1354443Z ##[section]Starting: Checkout
2019-09-18T10:35:36.1465943Z ==============================================================================
2019-09-18T10:35:36.1466042Z Task         : Get sources
2019-09-18T10:35:36.1466294Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-09-18T10:36:43Z

💔 Test failed - checks-azure

Looks like `script`, which uses `cmd.exe`, doesn't have fail-fast behavior and if a leading command fails the script doesn't actually fail so long as the last command succeeds. We instead want the opposite behavior where if any step fails the whole script fails. I don't really know `cmd.exe` that well, nor powershell, so I've opted to move everything to `bash` which should be a good common denominator amongst all platforms to work with. Additionally I know that `set -e` works to cause scripts to fail fast. Note that some scripts remain as `script` since they don't appear to work in` bash`. I'm not really sure why but I reorganized them slightly to have the "meaty command" run at the end.

alexcrichton · 2019-09-19T14:31:09Z

@bors: r=Mark-Simulacrum

bors · 2019-09-19T14:31:11Z

📌 Commit 72ea960 has been approved by Mark-Simulacrum

bors · 2019-09-20T11:39:04Z

⌛ Testing commit 72ea960 with merge 9ad1e7c...

…ark-Simulacrum azure: Convert Windows installations scripts to `bash` Looks like `script`, which uses `cmd.exe`, doesn't have fail-fast behavior and if a leading command fails the script doesn't actually fail so long as the last command succeeds. We instead want the opposite behavior where if any step fails the whole script fails. I don't really know `cmd.exe` that well, nor powershell, so I've opted to move everything to `bash` which should be a good common denominator amongst all platforms to work with. Additionally I know that `set -e` works to cause scripts to fail fast. Closes #64551

bors · 2019-09-20T15:35:45Z

☀️ Test successful - checks-azure
Approved by: Mark-Simulacrum
Pushing 9ad1e7c to master...

rust-highfive assigned Mark-Simulacrum Sep 17, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 17, 2019

Mark-Simulacrum approved these changes Sep 17, 2019

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 17, 2019

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 17, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 17, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 18, 2019

alexcrichton force-pushed the windows-bash-install-scripts branch from eb68333 to 72ea960 Compare September 19, 2019 14:30

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 19, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 20, 2019

bors merged commit 72ea960 into rust-lang:master Sep 20, 2019

alexcrichton deleted the windows-bash-install-scripts branch September 20, 2019 16:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

azure: Convert Windows installations scripts to `bash` #64553

azure: Convert Windows installations scripts to `bash` #64553

alexcrichton commented Sep 17, 2019 •

edited

Loading

rust-highfive commented Sep 17, 2019

Mark-Simulacrum Sep 17, 2019

alexcrichton Sep 17, 2019

Mark-Simulacrum commented Sep 17, 2019

bors commented Sep 17, 2019

Mark-Simulacrum commented Sep 17, 2019

alexcrichton commented Sep 17, 2019

bors commented Sep 17, 2019

bors commented Sep 18, 2019

rust-highfive commented Sep 18, 2019

bors commented Sep 18, 2019

alexcrichton commented Sep 19, 2019

bors commented Sep 19, 2019

bors commented Sep 20, 2019

bors commented Sep 20, 2019

azure: Convert Windows installations scripts to bash #64553

azure: Convert Windows installations scripts to bash #64553

Conversation

alexcrichton commented Sep 17, 2019 • edited Loading

rust-highfive commented Sep 17, 2019

Mark-Simulacrum Sep 17, 2019

Choose a reason for hiding this comment

alexcrichton Sep 17, 2019

Choose a reason for hiding this comment

Mark-Simulacrum commented Sep 17, 2019

bors commented Sep 17, 2019

Mark-Simulacrum commented Sep 17, 2019

alexcrichton commented Sep 17, 2019

bors commented Sep 17, 2019

bors commented Sep 18, 2019

rust-highfive commented Sep 18, 2019

bors commented Sep 18, 2019

alexcrichton commented Sep 19, 2019

bors commented Sep 19, 2019

bors commented Sep 20, 2019

bors commented Sep 20, 2019

azure: Convert Windows installations scripts to `bash` #64553

azure: Convert Windows installations scripts to `bash` #64553

alexcrichton commented Sep 17, 2019 •

edited

Loading