ICE on macro expansion in nightly, but not beta.

[Sean1708]

I can't really pin down the exact cause of this, but a MWE is here. Compiling on beta produces a normal error, but the nightly compiler panics.

On my machine:

$ cat main.rs
#[macro_export]
macro_rules! parallel {
    (
        $( let $priv_id:ident = $ex:expr; )*
        for $id:ident in $iter:expr {
            $( $inner:expr; )*
        }
    ) => {{
        $( let mut $priv_id = $ex; )*
        for $id in $iter {
            $( $inner; )*
        }
    }};
}


fn main() {
    let n = 10000;

    parallel! {
        let x = 0;
        for i in 0..n {
            x += i;
        }
    }
}
$ multirust --version
multirust 0.7.0 (b222fcd27 2015-09-24 10:53:38 -0700)
$ multirust run beta rustc --version
rustc 1.6.0-beta.1 (2e48b593c 2015-12-09)
$ multirust run beta rustc main.rs
main.rs:23:15: 23:17 error: expected `:`, found `+=`
main.rs:23             x += i;
                         ^~
main.rs:23:15: 23:17 error: no rules expected the token `+=`
main.rs:23             x += i;
                         ^~
$ multirust run nightly rustc --version
rustc 1.7.0-nightly (d5e229057 2016-01-04)
$ RUST_BACKTRACE=1 multirust run nightly rustc main.rs
error: internal compiler error: Error constructed but not emitted
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
note: run with `RUST_BACKTRACE=1` for a backtrace
stack backtrace:
   1:        0x10a012d68 - sys::backtrace::tracing::imp::write::h766e1257c1f84926B0t
   2:        0x10a01911f - panicking::default_handler::_<closure>::closure.42357
   3:        0x10a018b62 - panicking::default_handler::h68d3e445172abaccSXx
   4:        0x109fe0c34 - sys_common::unwind::begin_unwind_inner::hc0cc487f54c9cc74D3s
   5:        0x109731234 - sys_common::unwind::begin_unwind::h13547604483651865760
   6:        0x1097383ff - errors..DiagnosticBuilder::drop.55913::h960f3327a38997ec
   7:        0x109867e13 - parse::parser::Parser<'a>::parse_assoc_expr_with::h54bfe9d5867b9a2atNK
   8:        0x109a482ac - ext::tt::macro_parser::parse_nt::ha045d533d9473e78JUd
   9:        0x109779628 - ext::tt::macro_parser::parse::h3081a47c878f0d47yEd
  10:        0x109776522 - ast::TokenTree::parse::hff59eeb2ee1d439eMoo
  11:        0x109a4b729 - ext::tt::macro_rules::MacroRulesMacroExpander.TTMacroExpander::expand::h9612813d3dcdb29c2he
  12:        0x109998703 - ext::expand::expand_expr::_<closure>::closure.67968
  13:        0x109996db7 - ext::expand::expand_expr::hae69d05227aa6495ZH9
  14:        0x1099e0ff9 - option::Option<T>::map::h15133355466387097982
  15:        0x1099dfe92 - ptr::P<T>::map::h8018096219495759744
  16:        0x1099dfc19 - ext::expand::expand_block::hd439b735ebedccdb8ja
  17:        0x1099bc3f6 - ext::expand::expand_and_rename_fn_decl_and_block::h5ffad287890948b2dIa
  18:        0x1099c9c46 - ext::expand::expand_item_underscore::h21fd83f940aaf048jY9
  19:        0x109a004ba - fold::noop_fold_item_simple::h11114274549110489552
  20:        0x109a00059 - fold::noop_fold_item::h14585974302004177293
  21:        0x1099c7caa - ext::expand::expand_annotatable::h8c7d426b1e137197Zua
  22:        0x1099c4058 - ext::expand::expand_item::h39bd2724f5135651IX9
  23:        0x1099d0379 - fold::noop_fold_mod::h1378000857864055771
  24:        0x1099ca411 - ext::expand::expand_item_underscore::h21fd83f940aaf048jY9
  25:        0x109a004ba - fold::noop_fold_item_simple::h11114274549110489552
  26:        0x109a00059 - fold::noop_fold_item::h14585974302004177293
  27:        0x1099c89e7 - ext::expand::expand_annotatable::h8c7d426b1e137197Zua
  28:        0x1099c4058 - ext::expand::expand_item::h39bd2724f5135651IX9
  29:        0x109a099d7 - ext::expand::expand_crate::h2fc8fffdacba03543Sa
  30:        0x105fc49f3 - driver::phase_2_configure_and_expand::_<closure>::closure.31431
  31:        0x105f5ba1f - driver::phase_2_configure_and_expand::h7778ea873c616208kya
  32:        0x105f3b052 - driver::compile_input::ha0a3752393232affjca
  33:        0x105f2ea7f - run_compiler::hc39777399ed593a2Lwc
  34:        0x105f2b77c - sys_common::unwind::try::try_fn::h14272627946465103852
  35:        0x10a010898 - __rust_try
  36:        0x10a00865e - sys_common::unwind::try::inner_try::hb4e0e5e0be40f8b5b0s
  37:        0x105f2bb51 - boxed::F.FnBox<A>::call_box::h8681487165422399260
  38:        0x10a017ebd - sys::thread::Thread::new::thread_start::h513501a6a141d3f0fdx
  39:     0x7fff8f919c12 - _pthread_body
  40:     0x7fff8f919b8f - _pthread_start
thread 'rustc' panicked at 'explicit panic', ../src/libsyntax/errors/mod.rs:261

[jonas-schievink]

cc @nrc since this looks related to your "Structured Errors" PR

[pnkfelix]

I don't know if anyone cares, but that macro is going to get broken for a different reason when the new macro future proofing lands (rust-lang/rfcs#1384 and #30450 )

[Sean1708]

The macro was already broken to be fair, but thanks for the heads up.

[pnkfelix]

Its also worth noting that the reason you get that error about "unexpected +=" is, I believe, because the macro matching does not have the same fine grain classes that the parser does. In particular, for ident in expr { ... } is not actually putting arbitrary expr in there, but rather a subset of expressions that cannot end in { ... }. (Examples of the latter include, in particular, struct literals.)

Macros do not currently have the ability to express that subclass of expressions.

---

So when the pattern matches for $id:ident in $iter:expr { ... } against for i in 0..n { x += 1; }, it ends up trying to parse the latter as

for i in (0..(n { x += 1; }))

(That's right: A range expression, where the high end of the range is a struct literal. Hey, that's what you get when you have such a broad range of expression forms!) And then it fails the parse when it sees a += after x, when it is expecting a : that you get in struct literal expressions S { name: value, ... }

---

But I think the $moelarry idea from rust-lang/rfcs#1384 (see rust-lang/rfcs#1384 (comment) ) would I think provide that functionality (of specifying the narrower class of expressions that the user actually wants to match as the expr following the in token). Maybe this is the example we've been looking for to motivate adding that new fragment class.