Some fixes for error recovery in the compiler

[nrc]

No description provided.

[rust-highfive]

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

[sanxiyn]

Travis failed.

failures:
    [parse-fail] parse-fail/issue-10636-2.rs
    [parse-fail] parse-fail/issue-2354-1.rs
    [parse-fail] parse-fail/macro-mismatched-delim-paren-brace.rs

[nikomatsakis]

is this code specific to macros? if so, I don't see any tests below that would exercise it.

[nikomatsakis]

I realize now this was targeting primarily #31804 -- does it also aim to address #31994 ?

[nikomatsakis]

I feel like once we emit one unclosed delimeter error, we should try to avoid reporting any further ones, since all further delimeters are pretty suspect.

[nikomatsakis]

@eddyb tells me that we parse all things as token trees first -- I didn't realize that! Disregard my comments, will re-review.

[nikomatsakis]

Just for reference, the output from @brson's test in #31994 is https://gist.github.com/nikomatsakis/e6058b1e264e28d4885c -- still quite a lot. WRONG!

[nikomatsakis]

Oops, totally ran that with the wrong branch. The output is:

/home/nmatsakis/tmp/error-spew.rs:843:9: 843:10 error: incorrect close delimiter: `}`
/home/nmatsakis/tmp/error-spew.rs:843         } else {
                                              ^
/home/nmatsakis/tmp/error-spew.rs:841:21: 841:22 note: unclosed delimiter
/home/nmatsakis/tmp/error-spew.rs:841             callback(path.as_ref(); // MISSING )
                                                          ^
/home/nmatsakis/tmp/error-spew.rs:841:35: 841:36 error: expected one of `,`, `.`, `?`, or an operator, found `;`
/home/nmatsakis/tmp/error-spew.rs:841             callback(path.as_ref(); // MISSING )
                                                                        ^
/home/nmatsakis/tmp/error-spew.rs:842:13: 842:15 error: expected one of `.`, `;`, `?`, `}`, or an operator, found `fs`
/home/nmatsakis/tmp/error-spew.rs:842             fs::create_dir_all(path.as_ref()).map(|()| true)
                                                  ^~
/home/nmatsakis/tmp/error-spew.rs:841:13: 841:36 error: mismatched types:
 expected `core::result::Result<bool, std::io::error::Error>`,
    found `()`
(expected enum `core::result::Result`,
    found ()) [E0308]
/home/nmatsakis/tmp/error-spew.rs:841             callback(path.as_ref(); // MISSING )
                                                  ^~~~~~~~~~~~~~~~~~~~~~~
/home/nmatsakis/tmp/error-spew.rs:841:13: 841:36 help: run `rustc --explain E0308` to see a detailed explanation
error: aborting due to previous error

[nikomatsakis]

@bors r+

[bors]

📌 Commit 180d6b5 has been approved by nikomatsakis

[nikomatsakis]

Seems good to me. :)

[pnkfelix]

discussion leads to conclusion that we will land
only first commit from #32435 (3ee841c)
since #32494 will cover the spew otherwise
for beta