Allow rustdoc to handle asm! of foreign architectures #82838

Amanieu · 2021-03-06T16:17:43Z

This allows rustdoc to process code containing asm! for architectures other than the current one. Since this never reaches codegen, we just replace target-specific registers and register classes with a dummy one.

Fixes #82869

rust-highfive · 2021-03-06T16:17:46Z

r? @davidtwco

(rust-highfive has picked a reviewer for you, use r? to override)

jyn514 · 2021-03-06T17:38:13Z

Why does this make it to rustdoc at all? Special casing it with actually_rustdoc doesn't seem right, rustdoc shouldn't need to see the asm! in the first place because it doesn't typeck bodies:

rust/src/librustdoc/core.rs

Lines 396 to 405 in 51748a8

    
           // Certain queries assume that some checks were run elsewhere 
        
           // (see https://github.com/rust-lang/rust/pull/73566#issuecomment-656954425), 
        
           // so type-check everything other than function bodies in this crate before running lints. 
        
           // NOTE: this does not call `tcx.analysis()` so that we won't 
        
           // typeck function bodies or run the default rustc lints. 
        
           // (see `override_queries` in the `config`) 
        
           // HACK(jynelson) this calls an _extremely_ limited subset of `typeck` 
        
           // and might break if queries change their assumptions in the future.

jyn514 · 2021-03-06T17:38:59Z

compiler/rustc_ast_lowering/src/expr.rs

+        if self.sess.asm_arch.is_none() && !self.sess.opts.actually_rustdoc {
            struct_span_err!(self.sess, sp, E0472, "asm! is unsupported on this target").emit();
        }
        if asm.options.contains(InlineAsmOptions::ATT_SYNTAX)
            && !matches!(
                self.sess.asm_arch,
                Some(asm::InlineAsmArch::X86 | asm::InlineAsmArch::X86_64)
            )
+            && !self.sess.opts.actually_rustdoc
        {
            self.sess
                .struct_span_err(sp, "the `att_syntax` option is only supported on x86")


Why is this check done so early? I would expect it to be in codegen somewhere, or at least in MIR lowering.

jyn514 · 2021-03-06T17:42:04Z

Also, I'm not sure how this change helps with the stdarch failure:

error[E0428]: the name `__breakpoint` is defined multiple times
  --> crates/core_arch/src/arm/armclang.rs:49:1
   |
22 | pub unsafe fn __breakpoint<const VAL: i32>() {
   | -------------------------------------------- previous definition of the value `__breakpoint` here
...
49 | pub unsafe fn __breakpoint<const VAL: i32>() {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `__breakpoint` redefined here
   |
   = note: `__breakpoint` must be defined only once in the value namespace of this module

Amanieu · 2021-03-06T18:49:01Z

I've fixed the other issues in the stdarch PR. The latest CI results show the problem.

The main issue is the representation of register and register class constraints:

In the AST these are strings since macro expansion don't have access to target-specific information yet.
In the HIR these are target-specific enums representing an actual register or register class. This lowering is needed because typeck and asm validation (intrinsicck in rustc_passes) need to know the actual register class.

Even though rustdoc doesn't run the HIR passes it still performs AST lowering so we need to lower registers and register classes to a dummy type so we can still produce a valid HIR.

Amanieu · 2021-03-06T18:59:40Z

I've simplified the code: it turns out that rustdoc is happy enough if I just lower the entire asm! to hir::ExprKind::Err.

jyn514

It seems unfortunate to keep special-casing rustdoc everywhere, but I don't know how hard it would be move this later in the compiler. This seems no worse than the other hacks for rustdoc.

compiler/rustc_ast_lowering/src/expr.rs

nagisa · 2021-03-07T16:40:14Z

LGTM from the compiler impl standpoint.

@bors r+

I, too, agree that having to special-case rustdoc here is unfortunate, IMHO the best way to fix this would be to make it possible for rustdoc itself to entirely skip any handling of function bodies in any part of the compiler it ends up invoking. A most straightforward way would probably be walking the AST and replacing all the bodies with Err? (I for some reason remember that rustdoc did something along the lines of -Zeverybody-loops but that's not the case anymore?)

nagisa · 2021-03-07T16:42:52Z

@bors r+

bors · 2021-03-07T16:42:53Z

📌 Commit 4d2413296e5d849f75001128b2a9df026df6acdb has been approved by nagisa

jyn514 · 2021-03-07T16:45:57Z

A most straightforward way would probably be walking the AST and replacing all the bodies with Err? (I for some reason remember that rustdoc did something along the lines of -Zeverybody-loops but that's not the case anymore?)

This broke quite a lot of rustdoc, it was changed about a year ago: #73566. The way to do this properly is to move the error checking to after AST desugaring, when typechecking is being done, then rustdoc should ignore it automatically.

Amanieu · 2021-03-07T17:48:43Z

Now that you mention it, could this change cause issues since we aren't lowering the Exprs in the asm! operands?

jyn514 · 2021-03-07T18:05:40Z

Now that you mention it, could this change cause issues since we aren't lowering the Exprs in the asm! operands?

I think so, yes. Try this test case:

#![feature(decl_macro)]
#![feature(asm)]

pub unsafe fn aarch64(a: f64, b: f64) {
    let c;
    asm!("add {:d}, {:d}, d0", out(vreg) c, in(vreg) a, in("d0") {
        || {
            macro m() {}
        };
        b
    });
    c
}

jyn514 · 2021-03-07T18:06:21Z

lol that causes rustc itself to fail, even without these changes:

thread 'rustc' panicked at 'called `Option::unwrap()` on a `None` value', compiler/rustc_middle/src/hir/map/mod.rs:300:29
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

stack backtrace:
   0: rust_begin_unwind
             at /rustc/45b3c28518e4c45dfd12bc2c4400c0d0e9639927/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /rustc/45b3c28518e4c45dfd12bc2c4400c0d0e9639927/library/core/src/panicking.rs:92:14
   2: core::panicking::panic
             at /rustc/45b3c28518e4c45dfd12bc2c4400c0d0e9639927/library/core/src/panicking.rs:50:5
   3: rustc_middle::hir::map::Map::body_owner
   4: rustc_middle::hir::map::Map::body_owner_def_id
   5: rustc_middle::ty::<impl rustc_middle::ty::context::TyCtxt>::par_body_owners
   6: rustc_typeck::check::typeck_item_bodies
   7: rustc_query_system::dep_graph::graph::DepGraph<K>::with_task_impl
   8: rustc_data_structures::stack::ensure_sufficient_stack
   9: rustc_query_system::query::plumbing::force_query_with_job
  10: rustc_query_system::query::plumbing::get_query_impl
  11: <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::typeck_item_bodies
  12: rustc_session::utils::<impl rustc_session::session::Session>::time
  13: rustc_typeck::check_crate
  14: rustc_interface::passes::analysis
  15: rustc_query_system::dep_graph::graph::DepGraph<K>::with_task_impl
  16: rustc_data_structures::stack::ensure_sufficient_stack
  17: rustc_query_system::query::plumbing::force_query_with_job
  18: rustc_query_system::query::plumbing::get_query_impl
  19: <rustc_query_impl::Queries as rustc_middle::ty::query::QueryEngine>::analysis
  20: rustc_interface::passes::QueryContext::enter
  21: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter
  22: rustc_span::with_source_map
  23: rustc_interface::interface::create_compiler_and_run
  24: rustc_span::with_session_globals
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

jyn514 · 2021-03-07T18:08:52Z

Here's an example that compiles when you add --target aarch64-unknown-linux-gnu:

#![feature(decl_macro)]
#![feature(asm)]

pub unsafe fn aarch64(a: f64, b: f64) -> f64 {
    let c;
    asm!("add {:d}, {:d}, d0", out(vreg) c, in(vreg) a, in("d0") {
        || {
            macro m() {}
        };
        b
    });
    c
}

nagisa · 2021-03-07T18:11:38Z

asm!("...", in(reg) {
    /// documentation
    pub fn banana(...) { ... }
    42
})

kind of thing? Yeah, not lowering this would most likely be prone to causing all the same issues as what everybody-loops did.

@bors r-

Amanieu · 2021-03-08T01:15:12Z

I think the root cause is probably linked to #82869, so let's figure that out first.

Amanieu · 2021-03-09T03:49:39Z

I can see 2 ways of solving this properly:

Add an Invalid variant to InlineAsmReg and InlineAsmRegClass so we can still produce valid HIR even for asm! with invalid registers for the current target.
Defer all asm register & type validation to HIR->MIR lowering. HIR would then simply represent register names with Symbol like the AST does.

Amanieu · 2021-03-13T19:58:22Z

Updated, this should also fix #82869.

r? @nagisa

src/test/rustdoc-ui/asm-foreign.rs

nagisa

this should also fix #82869.

Can you please add a regression test for this as well? Overall LGTM! r=me.

nagisa · 2021-03-13T21:08:34Z

compiler/rustc_target/src/asm/mod.rs

@@ -309,6 +313,7 @@ impl InlineAsmReg {
            Self::RiscV(r) => r.emit(out, arch, modifier),
            Self::Hexagon(r) => r.emit(out, arch, modifier),
            Self::Mips(r) => r.emit(out, arch, modifier),
+            Self::Err => unreachable!(),


Many of these unreachable!s here and in {modifier,reg}_to_llvm feel like they should be a bug! instead. These aren't very obviously unreachable and in the future it might be the case that somebody does change some code in a way that would cause this to become reached too (thus introducing a bug).

bug! is defined in rustc_middle which rustc_target can't depend on since that would result in a circular dependency. I've added a message to the unreachable though.

compiler/rustc_ast_lowering/src/expr.rs

Amanieu · 2021-03-15T23:22:41Z

@bors r=nagisa

bors · 2021-03-15T23:22:43Z

📌 Commit ba00ddc has been approved by nagisa

Allow rustdoc to handle asm! of foreign architectures This allows rustdoc to process code containing `asm!` for architectures other than the current one. Since this never reaches codegen, we just replace target-specific registers and register classes with a dummy one. Fixes rust-lang#82869

bors · 2021-03-16T10:05:49Z

⌛ Testing commit ba00ddc with merge f24ce9b...

bors · 2021-03-16T12:47:00Z

☀️ Test successful - checks-actions
Approved by: nagisa
Pushing f24ce9b to master...

rust-highfive assigned davidtwco Mar 6, 2021

Amanieu force-pushed the rustdoc_asm branch from 15118ae to f927f15 Compare March 6, 2021 16:17

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 6, 2021

Amanieu mentioned this pull request Mar 6, 2021

Convert all uses of llvm_asm! to asm! rust-lang/stdarch#1052

Merged

jyn514 reviewed Mar 6, 2021

View reviewed changes

jyn514 added the T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. label Mar 6, 2021

Amanieu force-pushed the rustdoc_asm branch 2 times, most recently from bd60df1 to 4131e86 Compare March 6, 2021 18:58

Amanieu force-pushed the rustdoc_asm branch 2 times, most recently from 32cda4a to d5d6408 Compare March 6, 2021 19:00

jyn514 reviewed Mar 6, 2021

View reviewed changes

compiler/rustc_ast_lowering/src/expr.rs Outdated Show resolved Hide resolved

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 7, 2021

nagisa assigned nagisa and unassigned davidtwco Mar 7, 2021

bors removed the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Mar 7, 2021

bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Mar 7, 2021

jyn514 mentioned this pull request Mar 7, 2021

ICE on nested items when asm! has errors #82869

Closed

Amanieu force-pushed the rustdoc_asm branch from 4d24132 to d55e9dc Compare March 8, 2021 01:04

Amanieu mentioned this pull request Mar 13, 2021

make is_{arch}_feature_detected! available for std rust-lang/stdarch#1081

Closed

Amanieu force-pushed the rustdoc_asm branch from d55e9dc to cd4b9cd Compare March 13, 2021 19:57

This comment has been minimized.

Sign in to view

Always lower asm! to valid HIR

fa3694f

Amanieu force-pushed the rustdoc_asm branch from cd4b9cd to fa3694f Compare March 13, 2021 20:51

jyn514 reviewed Mar 13, 2021

View reviewed changes

src/test/rustdoc-ui/asm-foreign.rs Outdated Show resolved Hide resolved

nagisa approved these changes Mar 13, 2021

View reviewed changes

Address review comments

ba00ddc

Amanieu force-pushed the rustdoc_asm branch 2 times, most recently from 8f76dda to ba00ddc Compare March 14, 2021 23:22

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 15, 2021

Dylan-DPC-zz mentioned this pull request Mar 16, 2021

Rollup of 11 pull requests #83178

Closed

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 16, 2021

bors merged commit f24ce9b into rust-lang:master Mar 16, 2021

rustbot added this to the 1.52.0 milestone Mar 16, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow rustdoc to handle asm! of foreign architectures #82838

Allow rustdoc to handle asm! of foreign architectures #82838

Amanieu commented Mar 6, 2021 •

edited

Loading

rust-highfive commented Mar 6, 2021

jyn514 commented Mar 6, 2021

jyn514 Mar 6, 2021

jyn514 commented Mar 6, 2021

Amanieu commented Mar 6, 2021

Amanieu commented Mar 6, 2021

jyn514 left a comment

nagisa commented Mar 7, 2021 •

edited

Loading

nagisa commented Mar 7, 2021

bors commented Mar 7, 2021

jyn514 commented Mar 7, 2021

Amanieu commented Mar 7, 2021

jyn514 commented Mar 7, 2021

jyn514 commented Mar 7, 2021

jyn514 commented Mar 7, 2021

nagisa commented Mar 7, 2021

Amanieu commented Mar 8, 2021

Amanieu commented Mar 9, 2021

Amanieu commented Mar 13, 2021

This comment has been minimized.

nagisa left a comment

nagisa Mar 13, 2021

Amanieu Mar 14, 2021

Amanieu commented Mar 15, 2021

bors commented Mar 15, 2021

bors commented Mar 16, 2021

bors commented Mar 16, 2021

Allow rustdoc to handle asm! of foreign architectures #82838

Allow rustdoc to handle asm! of foreign architectures #82838

Conversation

Amanieu commented Mar 6, 2021 • edited Loading

rust-highfive commented Mar 6, 2021

jyn514 commented Mar 6, 2021

jyn514 Mar 6, 2021

Choose a reason for hiding this comment

jyn514 commented Mar 6, 2021

Amanieu commented Mar 6, 2021

Amanieu commented Mar 6, 2021

jyn514 left a comment

Choose a reason for hiding this comment

nagisa commented Mar 7, 2021 • edited Loading

nagisa commented Mar 7, 2021

bors commented Mar 7, 2021

jyn514 commented Mar 7, 2021

Amanieu commented Mar 7, 2021

jyn514 commented Mar 7, 2021

jyn514 commented Mar 7, 2021

jyn514 commented Mar 7, 2021

nagisa commented Mar 7, 2021

Amanieu commented Mar 8, 2021

Amanieu commented Mar 9, 2021

Amanieu commented Mar 13, 2021

This comment has been minimized.

nagisa left a comment

Choose a reason for hiding this comment

nagisa Mar 13, 2021

Choose a reason for hiding this comment

Amanieu Mar 14, 2021

Choose a reason for hiding this comment

Amanieu commented Mar 15, 2021

bors commented Mar 15, 2021

bors commented Mar 16, 2021

bors commented Mar 16, 2021

Amanieu commented Mar 6, 2021 •

edited

Loading

nagisa commented Mar 7, 2021 •

edited

Loading