Parsing syn::Pat inside of None-delimited group

[dodomorandi]

The minimal test I have been able to create until now is a bit convoluted. Create a lib crate with the following lib.rs:

#[cfg(test)]
macro_rules! implicit_gen_test {
    ($pat:pat) => {
        ::inner::gen_test!($pat);
    };
}

#[cfg(test)]
macro_rules! gen_test {
    ($pat:pat) => {
        implicit_gen_test!(($pat));
    };
}

#[test]
fn test_1_47_0_tokenstream() {
    gen_test!(Some(a));
}

Create the inner proc-macro crate with full syn and quote dependencies with the following lib.rs:

extern crate proc_macro;

use proc_macro::TokenStream;
use quote::quote;
use syn::{
    parse::{Parse, ParseStream},
    parse_macro_input, Pat, Result,
};

struct Test;

impl Parse for Test {
    fn parse(input: ParseStream) -> Result<Self> {
        input.parse::<Pat>()?;

        Ok(Self)
    }
}

#[proc_macro]
pub fn gen_test(input: TokenStream) -> TokenStream {
    let _: Test = parse_macro_input!(input);

    let output = quote! {
        ()
    };
    TokenStream::from(output)
}

The code successfully compiles with 1.46.0, but it breaks with 1.47.0. The main reason is the tokenstream is different because of this PR. I was discussing the problem in this issue, Firefox seems to suffer from a similar one.

I am posting this issue here because it is obviously a matter of correctly parsing the new tokenstream, but at the same time I am a bit concerned about the "double macro indirection" I am using to trigger the issue -- I don't have the minimum idea if the data in the tokenstream is sane or not. Removing the nested parentheses from the forwarding gen_test macro, or just removing the macros completely changes the scenario, breaking the compilation for older rustc versions or making everything go fine with newer ones.

If you think that syn is right not being able to parse that tokenstream as a valid AST, it would be probably helpful for the ecosystem to open some issues in crates like try_match that is using this sort of macro forwarding hack and now are broken. I will be happy to help in that direction if necessary.

[dtolnay]

Reduced:

macro_rules! inner {
    ($pat:pat) => { repro::repro!($pat) };
}

macro_rules! outer {
    ($pat:pat) => { inner!(($pat)) };
}

fn main() {
    outer!(T());
}

use proc_macro::TokenStream;
use syn::{parse_macro_input, Pat};

#[proc_macro]
pub fn repro(input: TokenStream) -> TokenStream {
    parse_macro_input!(input as Pat);
    TokenStream::new()
}