proc macro attributes lose span info when parsing, or never get passed span info, or something

[canndrew]

I'm not sure whether this is a bug in rustc, proc_macro2 or syn, or whether it's a single bug or multiple, but anyway...

Suppose I define the identity proc macro attribute like this:

#[proc_macro_attribute]
pub fn my_cool_macro(blah: TokenStream, body: TokenStream) -> TokenStream {
    println!("body == {:#?}", body);
    body
}

This proc macro works, and if there's any syntax errors in the macro-enclosed code then the compiler will be able to point to their correct location in source. I have no idea how it does this though since, weirdly, the spans in the debug-printed body are all #0 bytes(0..0).

Now supposed I parse the macro code and re-emit it as a new TokenStream.

#[proc_macro_attribute]
pub fn my_cool_macro(blah: TokenStream, body: TokenStream) -> TokenStream {
    println!("body == {:#?}", body);
    let file = syn::parse::<syn::File>(body).unwrap();
    let ret = file.into_token_stream().into();
    println!("ret == {:#?}", ret);
    ret
}

Even though this should still be the identity function, all the span information (which was apparently never there) gets lost. If there are syntax errors in the macro-enclosed code the compiler will instead just point at the #![my_cool_macro] attribute itself rather than the actual location of the error. This is reflected in the spans in ret which are all identical and all of the form #x bytes(y..(y + 17)) (where 17 is "#![my_cool_macro]".len()).

I'm very confused about what's going on here, where the problem is, and whether there's a workaround which will allow my macro to not break error reporting. Can anyone shine light on this and is there already a bug report tracking it?

[alexcrichton]

This is likely to be rust-lang/rust#43081 which is a sort of obtuse answer to this question. What's happening there is that rustc doesn't actually have the ability to convert an arbitrary item (like a function) to a TokenStream. Sometimes it'll do a fallback where it pretty-prints the function to a string and then parses that to a token stream, but this loses all span information!

If you don't modify the TokenStream (e.g. pass it through) then rustc doesn't do stringification, but if you convert it to an iterator of tokens then rustc will stringify.

What does the code look like that you're attaching the attribute to? There's likely a bug in rustc that's causing it to stringify too eagerly and/or something we should implement to make it not stringify. There's a lot of cc'd bugs on rust-lang/rust#43081 which also show weird behavior.

[canndrew]

In my case I'm using an attribute macro on a entire file, and it seems to be no matter what I put in the file it drops span information.

[alexcrichton]

You can try to debug this by printing the file with rustc -Z unstable-options --pretty normal, and if that file looks different than the one you fed in, that's the bug

[dtolnay]

Closing in favor of rust-lang/rust#43081.