Add proc-macro feature to control the runtime dependency on rustc toolchain libs

[staktrace]

@dtolnay Would you consider merging something like this?

Root problem is rust-lang/rust#47931 - my patch at rust-lang/rust#48217 would have solved it but is stalled. So as a workaround to get unblocked we extracted libproc_macro and published it as a standalone rustc-ap-proc_macro crate on crates.io, similar to how other libraries inside rustc are published as standalone crates (although I did it manually instead of scripting it, see alexcrichton/rustc-auto-publish#1). We then also forked proc-macro2, quote, and syn to standalone variants (now on crates.io as standalone-proc-macro2, standalone-quote, and standalone-syn) and updated cbindgen to use those. This solved our immediate problem, but is a pretty bad solution overall.

To make it a little better, it should be possible to just have a feature flag on proc-macro2, quote, and syn that does the same job, instead of having to fork the entire crate. This PR shows what the feature would look like for quote - if this looks acceptable to you I can do one for syn as well. Thoughts/comments?

Note that I haven't verified this actually does what I want it to (in terms of making cbindgen use this feature) but if you have no objections to the general idea then I can do that prior to merging this.

[alexcrichton]

FWIW my preferred route to solve this is here - dtolnay/proc-macro2#65 - notably that syn/quote would reexport the ability to turn off the linking to proc_macro

[dtolnay]

I cannot accept a standalone feature because Cargo features are required to be additive -- as implemented, this would break the ability to use both standalone and non-standalone quote in the same dependency graph. For example code like the following would break in confusing ways after adding a transitive dependency on some crate that uses standalone quote.

let tokens: proc_macro2::TokenStream = /* ... */;
syn::parse2(tokens)

I published Alex's PR as proc-macro2 0.2.3. Please update this PR to use that approach instead.

[staktrace]

@dtolnay makes sense, thanks for the explanation! I've updated the PR to use proc-macro2 0.2.3 and add a default-enabled proc-macro feature. Let me know if it looks good. I have a similar one ready for syn but I need a version number of quote published on crates.io before I can finalize that patch. I can still put up the WIP PR if you want.

[dtolnay]

Thanks! This approach seems reasonable to me.

One obstacle with doing the equivalent thing in Syn is that lots of libraries depend on syn = { version = "0.12", default-features = false, features = ["derive", "parsing"] } and expect to be able to call syn::parse which takes a proc_macro type as input. That code would break if syn::parse were to require an additional feature.

If possible I would prefer to introduce these "proc-macro" flags only in the next breaking release. What do you think?

[staktrace]

Yup that sounds reasonable. Can we just make the introduction of proc-macro come with a breaking version bump? Or did you want to wait until there's enough "breaking changes" accumulated before bumping the version?

Edit: fix typo

[dtolnay]

Let's batch it up with the next round of breaking changes. These updates tend to be a large amount of work for a project like Servo where they need to coordinate an update across dozens of repos. We try to make it worth their time when it happens. The best time may be when the next round of libproc_macro breaking changes rolls out -- tracked in rust-lang/rust#47786.

Could you make a Syn PR as well so we have that ready to go?

[staktrace]

Syn PR is at dtolnay/syn#363

[dtolnay]

Thanks! I continue to be on board with this change but we are not prepared to do a round of breaking changes right now. I filed #63 to make sure we reopen and merge this PR when ready. Until then I would recommend using a fork of quote published under a different name.