-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: switch macOS builders to 10.15 #68863
Conversation
@bors try Let's see what I screwed up. |
⌛ Trying commit ef0bc07e87e4bc50efb54cd3740bad6fc653d6ee with merge bfd963b0b0834db14d326e520e5169b867c996c8... |
Your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem. Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
💔 Test failed - checks-azure |
Largely r=me once things start working here; not sure about the python3 change but I presume it's necessary? |
Yep, Python 2 (at least in that location) doesn't exist on 10.15 anymore. |
ef0bc07
to
95c55b2
Compare
@bors try |
ci: switch macOS builders to 10.15 Azure Pipelines is deprecating the macOS 10.13 image we're currently running, [and they plan to remove them](https://devblogs.microsoft.com/devops/removing-older-images-in-azure-pipelines-hosted-pools/) on March 23, 2020. This PR switches our macOS builders to macOS 10.15. r? @Mark-Simulacrum
The job Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
💔 Test failed - checks-azure |
95c55b2
to
aaf93ab
Compare
@bors try |
⌛ Trying commit aaf93ab1331f56186e3adb87d926ed262d6f2620 with merge 7572e5cd224068ffd42db9de6f831d93aabc2964... |
The job Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
💔 Test failed - checks-azure |
Have you been able to run all tests on 10.15? I'm having problems with Cargo's test suite failing because processes are getting killed by the OS. 75% of my azure runs have failed so far. It looks like some error with Gatekeeper, I see these messages in the logs: Logstaskgated 11 libsystem_pthread.dylib 0x00007fff6522fe65 _pthread_start + 148 taskgated 12 libsystem_pthread.dylib 0x00007fff6522b83b thread_start + 15 taskgated no system signature for unsigned /Users/eric/Proj/rust/cargo/target/debug/cargo[30241] taskgated close(3) err: 0 taskgated end request taskgated begin request: 3331, 27001 taskgated UNIX error exception: 2 opendirectoryd PID: 30240, Client: 'clang', exited with 0 session(s), 0 node(s) and 0 active request(s) opendirectoryd Trigger - cancelled opendirectoryd Finalizing pidinfo (30240) object - 0x7fccecb44d00 taskgated 0 Security 0x00007fff3a1fea43 Security::CommonError::LogBacktrace() + 107 taskgated 1 Security 0x00007fff3a1fed5d Security::UnixError::UnixError(int, bool) + 263 taskgated 2 Security 0x00007fff3a1fedb8 Security::UnixError::throwMe(int) + 36 taskgated 3 Security 0x00007fff3a101c56 Security::CodeSigning::KernelCode::identifyGuest(Security::CodeSigning::SecCode*, __CFData const**) + 1006 taskgated 4 Security 0x00007fff3a0df0fa Security::CodeSigning::SecCode::identify() + 58 taskgated 5 Security 0x00007fff3a0df514 Security::CodeSigning::SecCode::autoLocateGuest(__CFDictionary const*, unsigned int) + 122 taskgated 6 Security 0x00007fff3a0e55a6 SecCodeCopyGuestWithAttributes + 78 taskgated 7 taskgated 0x000000010119f3b7 taskgated + 13239 taskgated 8 taskgated 0x000000010119facb taskgated + 15051 taskgated 9 taskgated 0x000000010119fe67 taskgated + 15975 taskgated 10 taskgated 0x000000010119fed3 taskgated + 16083 taskgated 11 taskgated 0x00000001011a20ea taskgated + 24810 taskgated 12 taskgated 0x00000001011a1617 taskgated + 22039 opendirectoryd PID: 30005, Client: 'cargo', exited with 0 session(s), 0 node(s) and 0 active request(s) taskgated 13 taskgated 0x00000001011a0ce5 taskgated + 19685 syspolicyd GK process assessment: <-- (, ) opendirectoryd Trigger - cancelled syspolicyd Unable (errno: 2) to read file at for process path: library path: (null) taskgated 14 libsystem_pthread.dylib 0x00007fff6522fe65 _pthread_start + 148 opendirectoryd Finalizing pidinfo (30005) object - 0x7fccead43440 syspolicyd Dropping com.apple.syspolicy.Gatekeeper.Errors as it isn't used in any transform (not in the config or budgeted?) kernel build_userspace_exit_reason: illegal flags passed from userspace (some masked off) 0x141, ns: 9, code 0x8 syspolicyd Terminating process due to Gatekeeper rejection: 30005, taskgated 15 libsystem_pthread.dylib 0x00007fff6522b83b thread_start + 15 kernel Waking up reference: 66615 taskgated no signature for pid=30005 (cannot make code: UNIX[No such file or directory]) kernel Thread waiting on reference 66615 woke up taskgated end request kernel Sleep interrupted, signal 0x100 kernel Security policy would not allow process: 30005, /Users/eric/Proj/rust/cargo/target/cit/t1430/foo/target/debug/deps/foo The key part is "Terminating process due to Gatekeeper rejection". The exception just above it is also unique to this error. I tried to repro with compiletest, but I wasn't able to. I figure since the run-pass tests quickly generate a large number of new executables it would maybe have the same problem. I'm a little concerned because I have no idea how to fix this. I suspect this is just a bug in Gatekeeper. Cargo's test suite could maybe check for processes killed with sigkill and restart them? I'm not sure if rust's test suite will be impacted by this on azure (I don't think my free account can build and run rustc, so I can't experiment). |
Linking against
The issue (I believe) is that the Line 610 in 4ff8fb9
It looks like when this flag was added (to compiler-rt) it was set to |
7743930
to
625c9cd
Compare
Thanks @rye! Looking a bit into it, it makes sense that the failure was caused by that line/PR: my GitHub Actions branch (which has a mostly similar configuration and environment) was building fine before the last rebase, while it failed to build with the same error after I rebased on the latest master. Let's see if everything works fine with that patch. cc @tmiasko (author of #65241) and @alexcrichton (reviewer of #65241) |
@bors try |
⌛ Trying commit 592bc1b033b22acabaa4c6d1ff3d3b20b72a0201 with merge 6bddff650254c446c55efe5f4ab1d3725c5493cc... |
☀️ Try build successful - checks-azure |
592bc1b
to
79c166e
Compare
Great! It works. r? @Mark-Simulacrum |
@bors r+ rollup=never (Feel free to prioritize, but I think we have enough slack time that it's hopefully not necessary). |
📌 Commit 79c166e has been approved by |
We actually did a try run with the affected builders and it was successful, so let's allow rolling this PR up. @bors rollup |
I would be concerned that without Cargo fixed, this will cause frequent failures. I'm not sure if the rust-lang macos machines are different enough to make much of a difference, but on the free microsoft hosted ones I saw a lot of errors. |
…ark-Simulacrum ci: switch macOS builders to 10.15 Azure Pipelines is deprecating the macOS 10.13 image we're currently running, [and they plan to remove them](https://devblogs.microsoft.com/devops/removing-older-images-in-azure-pipelines-hosted-pools/) on March 23, 2020. This PR switches our macOS builders to macOS 10.15. r? @Mark-Simulacrum
@ehuss we don't really have a choice, as Azure Pipelines is dropping the macOS 10.13 images on March 23rd. |
That's a month away? We're working on a fix now, I don't expect it to take that long. If this merges now, there is a chance that the majority of PRs will fail. Have you done multiple tests on the rust-lang builders to see what the error rate is? |
I'd prefer to have things in place in advance, to avoid rushing things in the last minute.
We did a couple of try builds in this PR and never got such failure (neither in the GHA configuration, which should be mostly the same). If this ends up causing spurious failures we can always revert. |
Rollup of 5 pull requests Successful merges: - #68863 (ci: switch macOS builders to 10.15) - #69142 (Add shared script for linkchecking books.) - #69248 (Don't eliminate frame pointers on thumb targets) - #69280 (Remove special case for `simd_shuffle` arg promotion) - #69284 (Reword OpenOptions::{create, create_new} doc.) Failed merges: r? @ghost
[beta] backports This backports the following PRs: * ci: switch macOS builders to 10.15 #68863 * Backport release notes of 1.41.1 #69468 * Cherry-pick the LLVM fix for #69225 #69450 * `lit_to_const`: gracefully bubble up type errors. #69330 * [beta] bootstrap from 1.41.1 stable #69518 * bootstrap: Configure cmake when building sanitizer runtimes #69104 r? @ghost
Azure Pipelines is deprecating the macOS 10.13 image we're currently running, and they plan to remove them on March 23, 2020. This PR switches our macOS builders to macOS 10.15.
r? @Mark-Simulacrum