Optimize parse+extend to make only one proc macro bridge call #203
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
proc_macro::TokenStream
provides bothExtend<TokenTree>
andExtend<TokenStream>
. They are implemented in libproc_macro as:For our use case in quote,
Extend<TokenTree>
is worse because it involves taking the TokenStream we parse (which is theimpl IntoIterator<Item = TokenTree>
passed into the Extend impl), iterating it to get a sequence of TokenTrees, and converting those TokenTrees back to a TokenStream. That's circuitous! We had a TokenStream already.This PR tweaks quote to use
Extend<TokenStream>
instead, which skips the needless round trip through TokenTree.This optimization is also important for dtolnay/proc-macro2#320. If the TokenStream holds its contents as a CallSite-spanned string buffer instead of trees, using
Extend<TokenTree>
forces proc-macro2 to reparse that buffered representation in order to iterate its tokens, whereasExtend<TokenStream>
allows the whole buffer to be efficiently appended to the lhs all at once.