Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make target-cpu=native detect individual features #80749

Merged
merged 3 commits into from
Jan 9, 2021

Conversation

as-com
Copy link
Contributor

@as-com as-com commented Jan 6, 2021

This PR makes target-cpu=native check for and enable/disable individual features instead of detecting and targeting a CPU by name. This brings the flag's behavior more in line with clang and gcc and ensures that the host actually supports each feature that we are compiling for.

This should resolve issues with miscompilations on e.g. "Haswell" Pentiums and Celerons that lack support for AVX, and also enable support for aes on Broadwell processors that support it. It should also resolve issues with failing to detect feature support in newer CPUs that aren't yet known by LLVM (see: #80633).

Fixes #54688
Fixes #48464
Fixes #38218

@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @oli-obk (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 6, 2021
@nagisa
Copy link
Member

nagisa commented Jan 7, 2021

r? @nagisa

This seems broadly LGTM, however I do have a concern with use of std::detect – in particular it only supports a few specific architectures (x86, x86_64, arm, aarch64, powerpc, powerpc64, mips and mips64). Furthermore it only iterates features that stdsimd knows about. This overall may lead to pessimizing the generated code.

With that in mind using LLVM's getHostCPUFeatures is probably better? It is also guaranteed that its output is suitable for passing to LLVM, avoiding the need to maintain a denylist of features like what you did with tsc here.

@rust-highfive rust-highfive assigned nagisa and unassigned oli-obk Jan 7, 2021
@as-com
Copy link
Contributor Author

as-com commented Jan 7, 2021

@nagisa Using getHostCPUFeatures would require an update to the C++ shim layer that translates the StringMap data structure. I'll take a look later to see if it's possible to do that without too much of a mess.

@nagisa
Copy link
Member

nagisa commented Jan 7, 2021

There's a C API of this as well LLVMGetHostCPUFeatures, that could let you avoid doing this, but all in all its no huge deal if you find it better to adjust rustc_llvm's RustWrapper to expose an API that is more convenient to consume from the Rust side (e.g. by returning string-length pairs rather than null-terminated strings).

@as-com
Copy link
Contributor Author

as-com commented Jan 8, 2021

Alright, I added LLVMGetHostCPUFeatures and LLVMDisposeMessage to ffi.rs, and updated the implementation accordingly. It works on my machine(tm) and should be more robust than the previous version.

@nagisa
Copy link
Member

nagisa commented Jan 8, 2021

@bors r+ rollup=iffy

Thanks!

@bors
Copy link
Contributor

bors commented Jan 8, 2021

📌 Commit 80ca198 has been approved by nagisa

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 8, 2021
@bors
Copy link
Contributor

bors commented Jan 8, 2021

⌛ Testing commit 80ca198 with merge 2d5c7a78104c9fbd6643bc6124bf1a1bda24ce55...

@bors
Copy link
Contributor

bors commented Jan 8, 2021

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 8, 2021
@rust-log-analyzer
Copy link
Collaborator

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@nagisa
Copy link
Member

nagisa commented Jan 8, 2021

@bors retry

Looks like a timeout

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 8, 2021
@JohnTitor
Copy link
Member

bors stuck?

@bors retry

@bors
Copy link
Contributor

bors commented Jan 9, 2021

⌛ Testing commit 80ca198 with merge c87ef0a...

@bors
Copy link
Contributor

bors commented Jan 9, 2021

☀️ Test successful - checks-actions
Approved by: nagisa
Pushing c87ef0a to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 9, 2021
@bors bors merged commit c87ef0a into rust-lang:master Jan 9, 2021
@rustbot rustbot added this to the 1.51.0 milestone Jan 9, 2021
wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request Apr 10, 2021
Pkgsrc changes:
 * Remove one SunOS patch, apparently no longer needed.
 * Adapt one patch for Darwin, adjust cargo checksum accordingly.
 * Adjust bootstraps to version 1.50.0.

Version 1.51.0 (2021-03-25)
============================

Language
--------
- [You can now parameterize items such as functions, traits, and
  `struct`s by constant values in addition to by types and
  lifetimes.][79135] Also known as "const generics" E.g. you can
  now write the following. Note:  Only values of primitive integers,
  `bool`, or `char` types are currently permitted.

  ```rust
  struct GenericArray<T, const LENGTH: usize> {
      inner: [T; LENGTH]
  }

  impl<T, const LENGTH: usize> GenericArray<T, LENGTH> {
      const fn last(&self) -> Option<&T> {
          if LENGTH == 0 {
              None
          } else {
              Some(&self.inner[LENGTH - 1])
          }
      }
  }
  ```

Compiler
--------

- [Added the `-Csplit-debuginfo` codegen option for macOS platforms.][79570]
  This option controls whether debug information is split across
  multiple files or packed into a single file. **Note** This option
  is unstable on other platforms.
- [Added tier 3\* support for `aarch64_be-unknown-linux-gnu`,
  `aarch64-unknown-linux-gnu_ilp32`, and
  `aarch64_be-unknown-linux-gnu_ilp32` targets.][81455]
- [Added tier 3 support for `i386-unknown-linux-gnu` and
  `i486-unknown-linux-gnu` targets.][80662]
- [The `target-cpu=native` option will now detect individual features
  of CPUs.][80749]
- [Rust now uses `inline-asm` for stack probes when used with LLVM
  11.0.1+][77885]

\* Refer to Rust's [platform support page][forge-platform-support]
for more information on Rust's tiered platform support.

Libraries
---------

- [`Box::downcast` is now also implemented for any `dyn Any + Send
  + Sync` object.][80945]
- [`str` now implements `AsMut<str>`.][80279]
- [`u64` and `u128` now implement `From<char>`.][79502]
- [`Error` is now implemented for `&T` where `T` implements `Error`.][75180]
- [`Poll::{map_ok, map_err}` are now implemented for `Poll<Option<Result<T,
  E>>>`.][80968]
- [`unsigned_abs` is now implemented for all signed integer types.][80959]
- [`io::Empty` now implements `io::Seek`.][78044]
- [`rc::Weak<T>` and `sync::Weak<T>`'s methods such as `as_ptr`
  are now implemented for `T: ?Sized` types.][80764]

Stabilized APIs
---------------

- [`Arc::decrement_strong_count`]
- [`Arc::increment_strong_count`]
- [`Once::call_once_force`]
- [`Peekable::next_if_eq`]
- [`Peekable::next_if`]
- [`Seek::stream_position`]
- [`array::IntoIter`]
- [`panic::panic_any`]
- [`ptr::addr_of!`]
- [`ptr::addr_of_mut!`]
- [`slice::fill_with`]
- [`slice::split_inclusive_mut`]
- [`slice::split_inclusive`]
- [`slice::strip_prefix`]
- [`slice::strip_suffix`]
- [`str::split_inclusive`]
- [`sync::OnceState`]
- [`task::Wake`]

Cargo
-----
- [Added the `split-debuginfo` profile option to control the -Csplit-debuginfo
  codegen option.][cargo/9112]
- [Added the `resolver` field to `Cargo.toml` to enable the new
  feature resolver and CLI option behavior.][cargo/8997] Version
  2 of the feature resolver will try to avoid unifying features of
  dependencies where that unification could be unwanted.  Such as
  using the same dependency with a `std` feature in a build scripts
  and proc-macros, while using the `no-std` feature in the final
  binary. See the [Cargo book documentation][[email protected]]
  for more information on the feature.

Rustdoc
-------
- [Rustdoc will now include documentation for methods available
  from `Deref` traits.][80653]
- [You can now provide a `--default-theme` flag which sets the
  default theme to use for documentation.][79642]

Various improvements to intra-doc links:

- [You can link to non-path primitives such as `slice`.][80181]
- [You can link to associated items.][74489]
- [You can now include generic parameters when linking to items,
  like `Vec<T>`.][76934]

Misc
----
- [You can now pass `--include-ignored` to tests (e.g. with
  `cargo test -- --include-ignored`) to include testing tests marked
  `#[ignore]`.][80053]

Compatibility Notes
-------------------

- [WASI platforms no longer use the `wasm-bindgen` ABI, and instead
  use the wasm32 ABI.][79998]
- [`rustc` no longer promotes division, modulo and indexing operations
  to `const` that could fail.][80579]

- [The minimum version of glibc for the following platforms has
  been bumped to version 2.31 for the distributed artifacts.][81521]
    - `armv5te-unknown-linux-gnueabi`
    - `sparc64-unknown-linux-gnu`
    - `thumbv7neon-unknown-linux-gnueabihf`
    - `armv7-unknown-linux-gnueabi`
    - `x86_64-unknown-linux-gnux32`

Internal Only
-------------

- [Consistently avoid constructing optimized MIR when not doing codegen][80718]

[79135]: rust-lang/rust#79135
[74489]: rust-lang/rust#74489
[76934]: rust-lang/rust#76934
[79570]: rust-lang/rust#79570
[80181]: rust-lang/rust#80181
[79642]: rust-lang/rust#79642
[80945]: rust-lang/rust#80945
[80279]: rust-lang/rust#80279
[80053]: rust-lang/rust#80053
[79502]: rust-lang/rust#79502
[75180]: rust-lang/rust#75180
[79135]: rust-lang/rust#79135
[81521]: rust-lang/rust#81521
[80968]: rust-lang/rust#80968
[80959]: rust-lang/rust#80959
[80718]: rust-lang/rust#80718
[80653]: rust-lang/rust#80653
[80579]: rust-lang/rust#80579
[79998]: rust-lang/rust#79998
[78044]: rust-lang/rust#78044
[81455]: rust-lang/rust#81455
[80764]: rust-lang/rust#80764
[80749]: rust-lang/rust#80749
[80662]: rust-lang/rust#80662
[77885]: rust-lang/rust#77885
[cargo/8997]: rust-lang/cargo#8997
[cargo/9112]: rust-lang/cargo#9112
[[email protected]]: https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2
[`Once::call_once_force`]: https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.call_once_force
[`sync::OnceState`]: https://doc.rust-lang.org/stable/std/sync/struct.OnceState.html
[`panic::panic_any`]: https://doc.rust-lang.org/stable/std/panic/fn.panic_any.html
[`slice::strip_prefix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix
[`slice::strip_suffix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix
[`Arc::increment_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.increment_strong_count
[`Arc::decrement_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.decrement_strong_count
[`slice::fill_with`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.fill_with
[`ptr::addr_of!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of.html
[`ptr::addr_of_mut!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of_mut.html
[`array::IntoIter`]: https://doc.rust-lang.org/nightly/std/array/struct.IntoIter.html
[`slice::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive
[`slice::split_inclusive_mut`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive_mut
[`str::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.str.html#method.split_inclusive
[`task::Wake`]: https://doc.rust-lang.org/nightly/std/task/trait.Wake.html
[`Seek::stream_position`]: https://doc.rust-lang.org/nightly/std/io/trait.Seek.html#method.stream_position
[`Peekable::next_if`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if
[`Peekable::next_if_eq`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if_eq
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request May 26, 2021
Pkgsrc changes:
 * Add support for the big-endian arm64 NetBSD target (aarch64_be).
 * On NetBSD/i386, use the i586 (pentium) bootstrap kit variant in
   preference to i686.
 * Adjust patches, re-compute line offsets, re-compute crate checksums.
 * Remove a patch which was either integrated upstream and/or no longer
   applies.
 * Bump bootstraps to 1.50.0.
 * Move conditionals until after bsd.prefs.mk so that they work...
 * Default to "dist" build target if cross-compiling, but allow
   also to override via rust.BUILD_TARGET.
 * Allow overriding MAKE_JOBS_SAFE via rust.MAKE_JOBS_SAFE if you
   want a different trade-off between occasional breakage and performance.
 * Adjust platform.mk according to work already done in wip/rust/
 * Add a patch to optimize the install.sh script used to install binary
   bootstraps to not do so many forks; use case/esac and parameter expansion
   instead of grep, sed and cut.
 * Drop building documentation for the binary bootstrap kits.  This will
   also impact the lang/rust-bin package.  For full documentation, build
   or install lang/rust as a package.

Upstream changes:

Version 1.51.0 (2021-03-25)
============================

Language
--------
- [You can now parameterize items such as functions, traits, and
  `struct`s by constant values in addition to by types and
  lifetimes.][79135] Also known as "const generics" E.g. you can
  now write the following. Note:  Only values of primitive integers,
  `bool`, or `char` types are currently permitted.
  ```rust
  struct GenericArray<T, const LENGTH: usize> {
      inner: [T; LENGTH]
  }

  impl<T, const LENGTH: usize> GenericArray<T, LENGTH> {
      const fn last(&self) -> Option<&T> {
          if LENGTH == 0 {
              None
          } else {
              Some(&self.inner[LENGTH - 1])
          }
      }
  }
  ```

Compiler
--------

- [Added the `-Csplit-debuginfo` codegen option for macOS platforms.][79570]
  This option controls whether debug information is split across
  multiple files or packed into a single file. **Note** This option
  is unstable on other platforms.
- [Added tier 3\* support for `aarch64_be-unknown-linux-gnu`,
  `aarch64-unknown-linux-gnu_ilp32`, and
  `aarch64_be-unknown-linux-gnu_ilp32` targets.][81455]

- [Added tier 3 support for `i386-unknown-linux-gnu` and
  `i486-unknown-linux-gnu` targets.][80662]

- [The `target-cpu=native` option will now detect individual features
  of CPUs.][80749]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------

- [`Box::downcast` is now also implemented for any `dyn Any + Send
  + Sync` object.][80945]
- [`str` now implements `AsMut<str>`.][80279]
- [`u64` and `u128` now implement `From<char>`.][79502]
- [`Error` is now implemented for `&T` where `T` implements `Error`.][75180]
- [`Poll::{map_ok, map_err}` are now implemented for
  `Poll<Option<Result<T,E>>>`.][80968]
- [`unsigned_abs` is now implemented for all signed integer types.][80959]
- [`io::Empty` now implements `io::Seek`.][78044]
- [`rc::Weak<T>` and `sync::Weak<T>`'s methods such as `as_ptr`
  are now implemented for `T: ?Sized` types.][80764]
- [`Div` and `Rem` by their `NonZero` variant is now implemented
  for all unsigned integers.][79134]

Stabilized APIs
---------------

- [`Arc::decrement_strong_count`]
- [`Arc::increment_strong_count`]
- [`Once::call_once_force`]
- [`Peekable::next_if_eq`]
- [`Peekable::next_if`]
- [`Seek::stream_position`]
- [`array::IntoIter`]
- [`panic::panic_any`]
- [`ptr::addr_of!`]
- [`ptr::addr_of_mut!`]
- [`slice::fill_with`]
- [`slice::split_inclusive_mut`]
- [`slice::split_inclusive`]
- [`slice::strip_prefix`]
- [`slice::strip_suffix`]
- [`str::split_inclusive`]
- [`sync::OnceState`]
- [`task::Wake`]
- [`VecDeque::range`]
- [`VecDeque::range_mut`]

Cargo
-----
- [Added the `split-debuginfo` profile option to control the -Csplit-debuginfo
  codegen option.][cargo/9112]
- [Added the `resolver` field to `Cargo.toml` to enable the new
  feature resolver and CLI option behavior.][cargo/8997] Version
  2 of the feature resolver will try to avoid unifying features of
  dependencies where that unification could be unwanted.  Such as
  using the same dependency with a `std` feature in a build scripts
  and proc-macros, while using the `no-std` feature in the final
  binary. See the [Cargo book documentation][[email protected]]
  for more information on the feature.

Rustdoc
-------

- [Rustdoc will now include documentation for methods available
  from _nested_ `Deref` traits.][80653]
- [You can now provide a `--default-theme` flag which sets the
  default theme to use for documentation.][79642]

Various improvements to intra-doc links:

- [You can link to non-path primitives such as `slice`.][80181]
- [You can link to associated items.][74489]
- [You can now include generic parameters when linking to items,
  like `Vec<T>`.][76934]

Misc
----
- [You can now pass `--include-ignored` to tests (e.g. with
  `cargo test -- --include-ignored`) to include testing tests marked
  `#[ignore]`.][80053]

Compatibility Notes
-------------------

- [WASI platforms no longer use the `wasm-bindgen` ABI, and instead
  use the wasm32 ABI.][79998]
- [`rustc` no longer promotes division, modulo and indexing operations
  to `const` that could fail.][80579]
- [The minimum version of glibc for the following platforms has
  been bumped to version 2.31 for the distributed artifacts.][81521]
    - `armv5te-unknown-linux-gnueabi`
    - `sparc64-unknown-linux-gnu`
    - `thumbv7neon-unknown-linux-gnueabihf`
    - `armv7-unknown-linux-gnueabi`
    - `x86_64-unknown-linux-gnux32`
- [`atomic::spin_loop_hint` has been deprecated.][80966] It's
  recommended to use `hint::spin_loop` instead.

Internal Only
-------------

- [Consistently avoid constructing optimized MIR when not doing codegen][80718]

[79135]: rust-lang/rust#79135
[74489]: rust-lang/rust#74489
[76934]: rust-lang/rust#76934
[79570]: rust-lang/rust#79570
[80181]: rust-lang/rust#80181
[79642]: rust-lang/rust#79642
[80945]: rust-lang/rust#80945
[80279]: rust-lang/rust#80279
[80053]: rust-lang/rust#80053
[79502]: rust-lang/rust#79502
[75180]: rust-lang/rust#75180
[79135]: rust-lang/rust#79135
[81521]: rust-lang/rust#81521
[80968]: rust-lang/rust#80968
[80959]: rust-lang/rust#80959
[80718]: rust-lang/rust#80718
[80653]: rust-lang/rust#80653
[80579]: rust-lang/rust#80579
[79998]: rust-lang/rust#79998
[78044]: rust-lang/rust#78044
[81455]: rust-lang/rust#81455
[80764]: rust-lang/rust#80764
[80749]: rust-lang/rust#80749
[80662]: rust-lang/rust#80662
[79134]: rust-lang/rust#79134
[80966]: rust-lang/rust#80966
[cargo/8997]: rust-lang/cargo#8997
[cargo/9112]: rust-lang/cargo#9112
[[email protected]]: https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2
[`Once::call_once_force`]: https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.call_once_force
[`sync::OnceState`]: https://doc.rust-lang.org/stable/std/sync/struct.OnceState.html
[`panic::panic_any`]: https://doc.rust-lang.org/stable/std/panic/fn.panic_any.html
[`slice::strip_prefix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix
[`slice::strip_suffix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix
[`Arc::increment_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.increment_strong_count
[`Arc::decrement_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.decrement_strong_count
[`slice::fill_with`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.fill_with
[`ptr::addr_of!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of.html
[`ptr::addr_of_mut!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of_mut.html
[`array::IntoIter`]: https://doc.rust-lang.org/nightly/std/array/struct.IntoIter.html
[`slice::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive
[`slice::split_inclusive_mut`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive_mut
[`str::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.str.html#method.split_inclusive
[`task::Wake`]: https://doc.rust-lang.org/nightly/std/task/trait.Wake.html
[`Seek::stream_position`]: https://doc.rust-lang.org/nightly/std/io/trait.Seek.html#method.stream_position
[`Peekable::next_if`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if
[`Peekable::next_if_eq`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if_eq
[`VecDeque::range`]: https://doc.rust-lang.org/nightly/std/collections/struct.VecDeque.html#method.range
[`VecDeque::range_mut`]: https://doc.rust-lang.org/nightly/std/collections/struct.VecDeque.html#method.range_mut
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
8 participants