Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rollup of 5 pull requests #93943

Closed
wants to merge 17 commits into from

Conversation

matthiaskrgr
Copy link
Member

Successful merges:

Failed merges:

r? @ghost
@rustbot modify labels: rollup

Create a similar rollup

abrown and others added 17 commits February 11, 2022 17:21
This change adds a flag for configuring control-flow protection in the
LLVM backend. In Clang, this flag is exposed as `-fcf-protection` with
options `none|branch|return|full`. This convention is followed for
`rustc`, though as a codegen option: `rustc -Z
cf-protection=<none|branch|return|full>`.

Co-authored-by: BlackHoleFox <[email protected]>
… r=oli-obk

More informative error message for E0015

Helps with rust-lang#92380
Add support for control-flow protection

This change adds a flag for configuring control-flow protection in the LLVM backend. In Clang, this flag is exposed as `-fcf-protection` with options `none|branch|return|full`. This convention is followed for `rustc`, though as a codegen option: `rustc -C cf-protection=<none|branch|return|full>`.

There are a couple of issues with this PR as-is, but I figured it would be advantageous to get some review feedback before proceeding much further:
 - [x] I would like this change would have some kind of test, e.g., compile a simple example with cf-protection enabled and verify that the ELF output indeed has the `IBT` and `SHSTK` issues; I will need someone to point out where such a test would go as I'm not too familiar with the `rustc` repository
 - [ ] Any assembly code in the libraries will need to be manually checked to see to it that when this flag is set, `ENDBR*` instructions are inserted in the right places; again, I will need a bit of help with this, since I don't know where all the x86 assembly lives [edit: see followup issue]
 - [x] A more serious issue, I think: if other libraries are not also built with these flags enabled, then LLVM will seek the "lowest-common denominator" and compile without cf-protection enabled; what I observed is that compiling a simple example with `-C cf-protection=full` doesn't result in the correct ELF tags, presumably because the standard libraries were not compiled with the same flags--any thoughts on this?
…rk-Simulacrum

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download

I got an error when trying to build the compiler using an old commit, and it turns out it was because the option `download-ci-llvm` was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem.

To reproduce my error:

```
git checkout 8d7707f
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
Build completed unsuccessfully in 0:00:46
```

This is my `config.toml`:

```
# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2

[rust]
debug = true
```

To reproduce an error with this branch:

Change line 618 of bootstrap.py to
```
        url = "rustc-builds-error404/{}".format(llvm_sha)
```

Delete llvm and cached tarball, and set `llvm.download-ci-llvm=true` in config.toml.

```
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
error: failed to download llvm from ci
help: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml:

[llvm]
download-ci-llvm = false

Build completed unsuccessfully in 0:00:01
```

Regarding the implementation, I expected to be able to use a try/catch block in `_download_ci_llvm`, but the `run` function calls `sys.exit` instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.
add link to format_args! when mention it in docs

close rust-lang#93904
update miri

to fix the libcore test suite
r? `@ghost`
@rustbot rustbot added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. rollup A PR which is a rollup labels Feb 12, 2022
@matthiaskrgr
Copy link
Member Author

@bors r+ rollup=never p=5

@bors
Copy link
Contributor

bors commented Feb 12, 2022

📌 Commit 555a6fe has been approved by matthiaskrgr

@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Feb 12, 2022
@bors
Copy link
Contributor

bors commented Feb 12, 2022

⌛ Testing commit 555a6fe with merge 6da423dba6f359b4552fec9a6697ee49e50dea1d...

@rust-log-analyzer
Copy link
Collaborator

The job aarch64-gnu failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
test [codegen] codegen/box-maybe-uninit.rs ... ok
test [codegen] codegen/debug-column-msvc.rs ... ignored
test [codegen] codegen/call-llvm-intrinsics.rs ... ok
test [codegen] codegen/c-variadic-opt.rs ... ok
test [codegen] codegen/cf-protection.rs#branch ... FAILED
test [codegen] codegen/cf-protection.rs#none ... FAILED
test [codegen] codegen/cf-protection.rs#full ... FAILED
test [codegen] codegen/cfguard-non-msvc.rs ... ok
test [codegen] codegen/dllimports/main.rs ... ignored
test [codegen] codegen/dllimports/main.rs ... ignored
test [codegen] codegen/cf-protection.rs#undefined ... FAILED
test [codegen] codegen/coercions.rs ... ok
test [codegen] codegen/cf-protection.rs#return ... FAILED
test [codegen] codegen/debug-linkage-name.rs ... ok
test [codegen] codegen/debug-compile-unit-path.rs ... ok
test [codegen] codegen/consts.rs ... ok
test [codegen] codegen/catch-unwind.rs ... ok
---
Some tests failed in compiletest suite=codegen mode=codegen host=aarch64-unknown-linux-gnu target=aarch64-unknown-linux-gnu

failures:

---- [codegen] codegen/cf-protection.rs#branch stdout ----

error in revision `branch`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "branch" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.branch" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "cf-protection=branch" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.branch/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------


---- [codegen] codegen/cf-protection.rs#none stdout ----

error in revision `none`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "none" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.none" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "cf-protection=none" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.none/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------


---- [codegen] codegen/cf-protection.rs#full stdout ----

error in revision `full`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "full" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.full" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "cf-protection=full" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.full/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------


---- [codegen] codegen/cf-protection.rs#undefined stdout ----

error in revision `undefined`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "undefined" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.undefined" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.undefined/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------


---- [codegen] codegen/cf-protection.rs#return stdout ----

error in revision `return`: compilation failed!
status: exit status: 1
command: "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/codegen/cf-protection.rs" "-Zthreads=1" "--cfg" "return" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.return" "-Crpath" "-O" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-Z" "cf-protection=return" "--target" "x86_64-unknown-linux-gnu" "-L" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/codegen/cf-protection.return/auxiliary" "--emit=llvm-ir"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`
  = help: consider building the standard library from source with `cargo build -Zbuild-std`
error: requires `sized` lang_item

error: aborting due to 2 previous errors


For more information about this error, try `rustc --explain E0463`.

------------------------------------------



failures:
    [codegen] codegen/cf-protection.rs#branch
    [codegen] codegen/cf-protection.rs#full
    [codegen] codegen/cf-protection.rs#none
    [codegen] codegen/cf-protection.rs#return
    [codegen] codegen/cf-protection.rs#undefined
test result: FAILED. 256 passed; 5 failed; 65 ignored; 0 measured; 0 filtered out; finished in 4.01s

Build completed unsuccessfully in 0:20:12

@bors
Copy link
Contributor

bors commented Feb 12, 2022

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 12, 2022
@matthiaskrgr matthiaskrgr deleted the rollup-wdajqnc branch February 13, 2022 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rollup A PR which is a rollup S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants