Expand documentation on procedural macros #405

alexcrichton · 2018-08-23T01:10:29Z

In preparation for the 1.30 stabilization I figured I'd get started and help
write some documentation!

alexcrichton · 2018-08-23T01:10:34Z

dtolnay · 2018-08-23T01:20:00Z

src/procedural-macros.md

+which is a far more stable interface over time for both the compiler and for
+procedural macros to target.
+
+A *token stream* is roughly equivalent to `Vec<TokenTree>` where a `TokenTree`


Somewhere it would be good to mention the key difference between TokenStream and Vec<TokenTree> -- that TokenStream is cheap to clone. Though maybe this belongs in rustdoc of the TokenStream type.

This definitely belongs in the library documentation and not the reference. We mostly just care that the type exists.

dtolnay · 2018-08-23T01:21:44Z

src/procedural-macros.md

+
+```rust,ignore
+#[proc_macro]
+pub fn foo(x: TokenStream) -> TokenStream {


Instead of x I would call it input.

I write the signature for procedural attributes in my branch like so:

Attribute macros are defined by a [public] function with the `proc_macro_attribute` attribute that a signature of `(TokenStream, TokenStream) -> TokenStream`.

dtolnay · 2018-08-23T01:29:34Z

src/procedural-macros.md

+#### Custom attributes on Custom derive
+
+An additional feature of custom derive macros is that they can whitelist names
+of attributes which are considered not part of normal attribute macro expansion.


We should pick a clear name for such attributes to distinguish them from attributes that are procedural macros. This is one of the most confusing aspects of the procedural macro API (practically everybody believes #[serde(...)] is itself a macro).

I have been using the terms "attribute macro" and "inert attribute" to distinguish the two.

I recently had to come up with a name for this in the compiler and used "derive helper attribute".

"Inert attribute", if it means non-macro attribute, an attribute that doesn't expand the item it's applied to, can currently refer to a built-in attribute (#[inline]), a tool attribute (#[rustfmt::skip]), a "derive helper" (#[serde(...)]), or a "custom" attribute on nightly (feature(custom_attribute) + #[my_attr]).

dtolnay · 2018-08-23T01:31:44Z

src/procedural-macros.md

-function. This function must have the type signature:
+* Custom macros - `my_macro!(...)`
+* Custom derive - `#[derive(MyTrait)]`
+* Custom attributes - `#[my_attribute]`


Let's kill the word "custom". It means nothing. The word should not appear on this page. We don't call things "custom crates" and "custom functions" and "custom structs".

I would write:

Function-like macros / bang macros

Derive macros

Attribute macros

dtolnay · 2018-08-23T01:36:32Z

src/procedural-macros.md

+
+```rust,ignore
+#[proc_macro_attribute]
+pub fn foo(attr: TokenStream, item: TokenStream) -> TokenStream {


The second argument is not necessarily an item if this attribute is on a statement or expression. I would name the two args and input. Their role is fully analogous to command-line arguments and command-line input such as in:

$ input | ./foo a b c

#[foo(a, b, c)] struct Input;

where a b c are arguments and separately there is input (the "in" in "stdin").

Custom attributes cannot be applied to non-items right now, so this is moot. If that changes before the final release, we can change it. I think args is a terrible name, even with the analogy. Though could be a good name for teaching. I opt for just showing the signature without naming them.

dtolnay · 2018-08-23T01:40:49Z

src/procedural-macros.md

+  This is primarily due to the lack of hygiene with procedural macros. Once a
+  better hygiene story is stabilized this restriction will likely be lifted.
+
+* Procedural macros cannot expand to definitions of `macro_rules!` macros (none


Hmm, serde_derive definitely does this now and it's stable. Maybe I misunderstand what this bullet means. Our two derives each include a macro_rules helper macro in the generated code that is called from elsewhere in the generated code.

This restriction is false for derive macros. Also missing that cannot expand to module items.

The restriction was implemented much later than derives were stabilized (rust-lang/rust#50820), so it was applied to fn-like macros and attribute macros to avoid breakage.

So macro_rules generated from derives do not produce immediate errors, but they can still be buggy and perhaps go through some breakage later with hygiene changes.

Havvy · 2018-08-23T04:31:14Z

I wish I knew you were working on this. I too have been working on it since Rustconf ended.

Havvy · 2018-08-23T04:32:39Z

src/procedural-macros.md

-that defines an interface for building a procedural macro. The
-`#[proc_macro_derive(Foo)]` attribute is used to mark the deriving
-function. This function must have the type signature:
+* Custom macros - `my_macro!(...)`


I thought function-like macros weren't being stabilized. Edit: I just tried it, and it works.

In preparation for the 1.30 stabilization I figured I'd get started and help write some documentation!

alexcrichton · 2018-08-24T21:30:54Z

Updated!

Havvy

Huh. I thought I submitted this two days ago...

This review is sort of weird because I was working on the same exact thing, and I want to merge them.

The main thing to note is that this page currently makes a great guide. But the reference isn't a guide, so quite a lot of this is useful w.r.t. the reference. So go publish this as is (or listen to @dtolnay first). Perhaps it can be modified into being the chapter on procedural macros in TRPL.

Overall, there's a lot of great information here that I would have had to spend a lot of time finding and figuring out how to actually write it. I marked the paragraphs that were especially helpful as "great" or "excellent". The general structure of sections is also pretty good. A little rearrangement to be referencey (e.g. cross-cutting things first instead of mostly at the end) is needed, but it's better than what I would have ultimately came up with.

A lot of comments were also to myself, since my plan of action with this is to remove guide and future speaking terms and then fit in the information from my own branch.

Havvy · 2018-08-23T04:43:43Z

src/procedural-macros.md

+
+### Crates and procedural macros
+
+All procedural macros in Rust are compiled as a crate. Procedural macro crates


in Rust is never necessary. Everything about the reference is about Rust unless specified otherwise.

Havvy · 2018-08-23T05:15:34Z

src/procedural-macros.md

+
+### Crates and procedural macros
+
+All procedural macros in Rust are compiled as a crate. Procedural macro crates


The first sentence doesn't really make sense. It's asserting that a procedural macro is a crate. Here's what I wrote in my branch: Procedural macros must be defined in a crate with the [crate type] of `proc_macro` with "crate type" linking to linkage.html.

The second sentence is Cargo-specific. And currently we have a policy about being compiler agnostic. That said, I did want to bend this rule and put this in a note. That can be done by starting the first line with > Note: and every other line with >.

Havvy · 2018-08-23T05:24:24Z

src/procedural-macros.md

+Procedural macros are always compiled with the same target as the compiler
+itself. For example if you execute `cargo build --target
+arm-unknown-linux-gnueabi` then procedural macros will not be compiled for ARM,
+but rather your build computer (for example `x86_64-unknown-linux-gnu`).


This paragraph is also less about procedural macros themselves and more about the proc_macro crate type. That said, this isn't currently documented about that crate type and it's not something I would have thought about to add either. So it's definitely good to have it documented somewhere. So want to move this to linkage.html?

We might want to move over the proc_macro crate type information to this page at some point, but for now, I'd like to keep all crate type information in linkage.html for now.

Havvy · 2018-08-23T05:49:49Z

src/procedural-macros.md

+facilities to working with the types of each procedural macro function. You can
+learn more about this crate by exploring [the documentation][pm-dox].
+
+[pm-dox]: https://doc.rust-lang.org/stable/proc_macro/


As per #391 we're moving to a consistent style of links all at the bottom of the page.

Havvy · 2018-08-23T05:50:18Z

src/procedural-macros.md

+
+### The `proc_macro` crate
+
+Procedural macro crates almost always will link to the in-tree `proc_macro`


I was struggling so hard with expressing this information. This entire paragraph is perfect! 😍 Well, almost. I'd replace "in-tree" with "compiler-provided" and remove "compiler-provided" from the second sentence.

Everything after this paragraph in this subsection is guide level and can be removed. The minimal examples for attribute macros and deriver macros should show that.

Havvy · 2018-08-23T08:02:00Z

src/procedural-macros.md

+Controlling spans is quite a powerful feature and needs to be used with care,
+misuse can lead to some excessively confusing error messages!
+
+### Procedural macros and hygiene


Good section. Think we can just call it "Hygiene" though. Should also move to before discussing the specific kinds of procedural macros.

Havvy · 2018-08-23T08:02:32Z

src/procedural-macros.md

+
+### Procedural macros and hygiene
+
+Currently all procedural macros are "unhygienic". This means that all procedural


This is close enough to a definition that we should put it in italics instead of quotes.

Havvy · 2018-08-23T08:09:06Z

src/procedural-macros.md

+  conservative limitation while compiler bugs are worked through and a design is
+  agreed upon.
+
+* Procedural attributes can only be attached to items, not expressions. For


This is really part of attributes.html. Might be useful to repeat in proc-macros.html anyways.

Havvy · 2018-08-23T08:09:42Z

src/procedural-macros.md

+  This is primarily due to the lack of hygiene with procedural macros. Once a
+  better hygiene story is stabilized this restriction will likely be lifted.
+
+* Procedural macros cannot expand to definitions of `macro_rules!` macros (none


This restriction is false for derive macros. Also missing that cannot expand to module items.

Havvy · 2018-08-23T08:10:03Z

src/procedural-macros.md

+  better hygiene story is stabilized this restriction will likely be lifted.
+
+* Procedural macros cannot expand to definitions of `macro_rules!` macros (none
+  of them). This restriction may be lifted over time but for now this is a


Talking about the future.

alexcrichton · 2018-09-03T17:57:59Z

Sorry @Havvy I've been really busy recently and haven't had a chance to come back to your comments here, but it looks like you've definitely got things in hand with #412, thanks for picking this up!

dtolnay reviewed Aug 23, 2018

View reviewed changes

Havvy reviewed Aug 23, 2018

View reviewed changes

Expand documentation on procedural macros

89800e0

In preparation for the 1.30 stabilization I figured I'd get started and help write some documentation!

alexcrichton force-pushed the macros branch from 1156a3d to 89800e0 Compare August 24, 2018 21:28

Havvy requested changes Aug 25, 2018

View reviewed changes

alexcrichton closed this Sep 3, 2018

alexcrichton deleted the macros branch September 3, 2018 17:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expand documentation on procedural macros #405

Expand documentation on procedural macros #405

alexcrichton commented Aug 23, 2018

alexcrichton commented Aug 23, 2018

dtolnay Aug 23, 2018

Havvy Aug 23, 2018

dtolnay Aug 23, 2018

Havvy Aug 23, 2018

dtolnay Aug 23, 2018

petrochenkov Aug 23, 2018

dtolnay Aug 23, 2018

Havvy Aug 23, 2018

dtolnay Aug 23, 2018

Havvy Aug 23, 2018

dtolnay Aug 23, 2018

Havvy Aug 23, 2018

petrochenkov Aug 23, 2018

Havvy commented Aug 23, 2018

Havvy Aug 23, 2018 •

edited

Loading

alexcrichton commented Aug 24, 2018

Havvy left a comment

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

Havvy Aug 23, 2018

alexcrichton commented Sep 3, 2018


		### Crates and procedural macros

		All procedural macros in Rust are compiled as a crate. Procedural macro crates


		### The `proc_macro` crate

		Procedural macro crates almost always will link to the in-tree `proc_macro`


		### Procedural macros and hygiene

		Currently all procedural macros are "unhygienic". This means that all procedural

Expand documentation on procedural macros #405

Expand documentation on procedural macros #405

Conversation

alexcrichton commented Aug 23, 2018

alexcrichton commented Aug 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Havvy commented Aug 23, 2018

Havvy Aug 23, 2018 • edited Loading

Choose a reason for hiding this comment

alexcrichton commented Aug 24, 2018

Havvy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexcrichton commented Sep 3, 2018

Havvy Aug 23, 2018 •

edited

Loading