Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow macros in types #873

Merged
merged 18 commits into from Jul 23, 2015
Merged

Allow macros in types #873

merged 18 commits into from Jul 23, 2015

Conversation

ghost
Copy link

@ghost ghost commented Feb 16, 2015

Allow macros in types (accepted)

rendered text/


Rendered draft

cc @aturon @huonw @jroesch

@jroesch
Copy link
Member

jroesch commented Feb 17, 2015

I think this would be very powerful. Making this type level fragment of the language more ergonomic would not only allow powerful functional programming constructs but the ability to compute information, and enforce powerful invariants at compile time.

You could now imagine parametrizing data types containing statically sized arrays like so:

struct Data<N: Nat> {
   internal_array: Array![Expr!(N + 1)] // or whatever the final syntax would be
}

I have recently wrote a file system in Rust and it would of been very cool to have the type system compute the necessary sizes of differing internal buffers based on parameters like BlockSize, # of blocks, # of INodes and so on.

@ghost
Copy link
Author

ghost commented Feb 17, 2015

Following the discussion on IRC, I added a little note about the possibility for optimization for Expr! using partial evaluation or other techniques.

@quantheory
Copy link
Contributor

After a couple of weeks of discussion, seeing this PR finally spurred me to open #884, which GitHub helpfully mentions right above this post. ^^^

That's not an alternative to this proposal per se, but I did want to point out that some of us have been looking at the approach of extending the type system, rather than relying on macros to write code that's generic over type-level numbers.

@ghost
Copy link
Author

ghost commented Feb 18, 2015

@quantheory Thanks for pointing out the other RFC. I've added a note to the type-level naturals example with a link to your RFC.

Essentially, I think these proposals are orthogonal, as you suggest. It just happens to be the case that type-level arithmetic (or type-level programming generally) that is currently possible in Rust becomes easier with macros in types. But I don't really intend for this to be a solution (not in any proper sense) to the more specific issue of parameterizing types by numeric values. I think allowing types in macros should just be generally useful, even if a more targeted solution gets implemented for that.

@jroesch
Copy link
Member

jroesch commented Feb 18, 2015

@quanttheory I'm reading over the discuss thread and the RFC this afternoon, but my initial thought is that by baking in arithmetic into the type system we add a lot of complexity without any generality (other type level values). It may very well be the most pragmatic way forward, but I think it is worth thinking about alternatives.

@quantheory
Copy link
Contributor

@jroesch My short answer is that if you add up the cases that already exist or are planned (constant expressions in arrays, associated const, sizeof) a significant chunk of the complexity is already present or expected to be brought in already for a much less general set of cases. So in that context I would consider the proposal an improvement in the overall complexity/expressiveness ratio, though that doesn't mean that there couldn't be something better.

That said, I do need some time to properly consider this and @freebroccolo's considered response. I would prefer to do any more discussion specific to the const parameter RFC on that request, just to avoid cluttering this PR with anything too off-topic.

@nrc
Copy link
Member

nrc commented Feb 20, 2015

What are the effects on hygiene? As I understand it there is no hygienic handling of types in macros at the moment, would allowing macros in type position make the hygiene situation worse? (I have no idea if there is some effect or not, but it seems worth thinking about).

@aturon
Copy link
Member

aturon commented Feb 20, 2015

@freebroccolo Just wanted to say, thanks for writing up this RFC! I've been busy with alpha2 but will make sure it gets attention.

cc @nikomatsakis

@ghost
Copy link
Author

ghost commented Feb 21, 2015

What are the effects on hygiene? As I understand it there is no hygienic handling of types in macros at the moment, would allowing macros in type position make the hygiene situation worse? (I have no idea if there is some effect or not, but it seems worth thinking about).

@nick29581 This is something I don't have a good answer for since I don't really understand the hygiene stuff in Rust that well. I did attempt to read some of the documentation and literature pointed to but I think someone who is more familiar with the internals will have to comment on that aspect.

@aturon
Copy link
Member

aturon commented Mar 5, 2015

ping @nikomatsakis @pnkfelix

@pnkfelix pnkfelix self-assigned this Mar 5, 2015
@pnkfelix
Copy link
Member

pnkfelix commented Mar 5, 2015

i will shepherd

@nikomatsakis
Copy link
Contributor

Hear ye, hear ye. This RFC was moved into Final Comment Period as of Wed, June 10th. (Sorry, forgot to add this notice!)

@nikomatsakis nikomatsakis added the final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. label Jun 11, 2015
@gsingh93
Copy link

I think you mean Wednesday, June 10th.

On Thu, Jun 11, 2015, 10:22 AM Niko Matsakis [email protected]
wrote:

Hear ye, hear ye. This RFC was moved into Final Comment Period as of
Wed, June 11th. (Sorry, forgot to add this notice!)


Reply to this email directly or view it on GitHub
#873 (comment).

@Mr-Byte
Copy link

Mr-Byte commented Jun 12, 2015

Seeing as how the current system is considered stable and the fact that this RFC would make current macros more useful in more places without really introducing new features to macros themselves (this is really just where they can be used) I'm in favor of this RFC.

I've had several use cases for needing macros in place of types, of which I've either hacked around with much nastier, more complex macros or I've given up on entirely and just wrote the code by hand. So I very much welcome this change.

@aturon
Copy link
Member

aturon commented Jun 15, 2015

While I share @brson's concern about feature creep (and also pity the type checker for the horrors this change is likely to unleash), I agree with others that this is a relatively minor change given the extant macro system. I don't think, for example, that it appreciably changes the difficulty that external tools already have to face when dealing with macros. And it's certainly a natural feature to want in conjunction with expression/item-level macros.

So on the whole, 👍 -- let's try this behind a feature gate, get some experience, and see what it looks like.

@jroesch
Copy link
Member

jroesch commented Jun 16, 2015

@aturon I am in support of this change as well and would be happy to help getting it in shape and merged assuming @freebroccolo doesn't want to do the work himself.

@ghost
Copy link
Author

ghost commented Jun 16, 2015

@jroesch if the change is accepted I would be glad for you work on the merge if you like. I kept my branch up to date with master for a few months but I haven't tried to build it recently so there's probably some breakage by now.

@paholg
Copy link

paholg commented Jun 16, 2015

I've been trying to figure out syntax extensions to make a convenient way to perform many type operations at once. If this goes through, I could do it in a couple lines in a regular macro.

I'm all for it.

@nrc
Copy link
Member

nrc commented Jun 16, 2015

AIUI, the argument for not doing this in the current macro system is that we could then allow it in a future one and that could be more of a carrot for using future macros rather than current ones. That is particularly appealing because future macros should be hygienic wrt to types and lifetimes and thus mean that this feature does not allow for the kind of badness that @nikomatsakis pointed out.

OTOH, I think it is a small and welcome extension to macro_rules and doesn't do any harm.

I do feel with every step like this we are improving things atm, at the expense of making adoption of some future macros system harder. That might be an OK trade off, but it is one we should make consciously.

@nikomatsakis
Copy link
Contributor

I just realized something about hygiene and lifetimes. This realization makes me feel both better and worse about the idea of macros that expand to types. Previously, I described how the lack of lifetime hygiene meant that if a macro introduced lifetimes binders in the form of a for<'a>... type, those names might interfere with other lifetime names in scope. But what I just remembered is shadowing lifetime names is currently a hard error: http://is.gd/oDoH6V

So this is not really a concern when it comes to forward compatibility, though it may be a usability concern for the macro author. I assume people will just limit themselves to expanding to awful things like for<'__foo_a> to achieve some primitive namespacing, but that won't work for recursive use of type macros. (Annoyingly, we also have no "lifetime" fragment specifier like $t:lifetime, so it's awkward to write macros that accept lifetimes and then use them within types.)

Just for completeness, the other interaction with hygiene is that type-expanding macros will likely want to refer to well-known types. e.g. if I wanted a macro myvec!{X} that expanded to MyVec<X>, I would have to write it to use something like $crate::MyVec<$X>, but I think this sort of thing is just a fact of life with today's macro system. It just happens that naming top-level items is going to be very, very common in macros that expand to types, and is somewhat less common for macros that expand to expressions.

@jroesch
Copy link
Member

jroesch commented Jun 18, 2015

@nikomatsakis would you be interested in expanding this to provide a fragment specifier for lifetimes? If it isn't out of scope I think it would be a useful/nice for type macros.

@retep998
Copy link
Member

Just wondering, would this allow macros in ident positions? For example struct SomeMacro!() {.

@nikomatsakis
Copy link
Contributor

@nrc

I do feel with every step like this we are improving things atm, at the expense of making adoption of some future macros system harder.

I'm not sure I fully understand what you mean. Do you mean "the more limited the current macro system is, the more tempting a new macro system will be to use?" That seems likely true -- if the current macro system is fulfilling most use cases people have in practice, it'll be more work to get them to migrate away.

That said, even if we did a great job selling a new, replacement macro system, I doubt we'll be able to remove the one we have. It is very widely used, that's why it's stable in the first place. I guess it's plausible that if the macro system is only being used in narrow ways, we might be able to keep some more restricted form around forever, but it's not clear what that subset is, or whether it would simplify the maintenance burden.

Or maybe you were thinking that we might be able to improve the existing macro system in place? For example, by adding lifetime hygiene? This also seems to get harder if macros have more use, but I strongly suspect the ship has largely sailed here, and the only way to do improve macros "in place" is with some sort of opt-in, whether that be a 2.0 release, a different form of macro_rules declaration, or something else.

For example, I know that the rusty_peg macro takes advantage of the lack of lifetime hygiene (admittedly, I only wrote things that way to work around the lack of lifetime hygiene). And I know of others that generate items parametrized by a few extra lifetimes (like 'a, 'b, 'c) "just in case" they are needed by the types the user writes. So I am quite confident that adding lifetime hygiene would break a reasonable number of existing macros.

@yongqli
Copy link

yongqli commented Jun 25, 2015

I would find this useful as well. My current use case is hacking around the lack of type level integers by asserting all the possible ways to multiply together matrices. For example:

impl<O: SquareMat, M, S: SquareMat> OpOn<O, S> for M
    where M: Mat<Row=O::Col, Col=S::Row> + Mul<S::Row, Output=O::Col> +
             Mul<S, Output=M> + Mul<<M as Mat>::Transpose, Output=O> {

The problem is that I have to assert all the possible ways to operate on all the matrices every place where I want to use a rectangular matrix. Being able to refactor them into a macro would be very nice. Ideally I would be able to write something like: op_on<O, M, S>(..) where rectangular_mat!(O, M, S) ...

@pythonesque
Copy link
Contributor

Given that it's the final comment period: very much in favor of this (I thought I'd already commented, but I guess not). It enables some really neat stuff.

@dylanede
Copy link

dylanede commented Jul 1, 2015

Very much in favour of this. However, @yongqli, I don't think you would be able to do what you suggest with impl bounds with this feature. This feature only enables macros in type positions, not where clauses.

@nikomatsakis
Copy link
Contributor

@jroesch

would you be interested in expanding this to provide a fragment specifier for lifetimes? If it isn't out of scope I think it would be a useful/nice for type macros.

I would be interested in having a fragment specific for lifetimes, yes. It does seem relevant to this RFC, but could also be separate.

@ghost
Copy link

ghost commented Jul 13, 2015

I have the same question as @retep998 - would this permit struct some_macro!() {?

@ghost
Copy link
Author

ghost commented Jul 14, 2015

@retep998 @Toby-s no, this doesn't change anything about top level identifier fragments in that regard.

@dylanede Actually, the implementation I provided was usable for where clauses, although there were some things that didn't quite work due (I assume) to somewhat incomplete/different handling of that part of the AST. In fact, I used it to make a fairly reasonable implementation of a haskell type-families style macro (e.g., it would derive implementations from lists of clauses and patterns).

@pnkfelix
Copy link
Member

@pnkfelix pnkfelix closed this Jul 23, 2015
@pnkfelix
Copy link
Member

(sorry that i rebased it to my own branch rather than merging, i'll try to fix that now so that the colors are nicer)

@pnkfelix pnkfelix reopened this Jul 23, 2015
pnkfelix added a commit that referenced this pull request Jul 23, 2015
@pnkfelix pnkfelix merged commit aac54c5 into rust-lang:master Jul 23, 2015
pnkfelix added a commit that referenced this pull request Jul 23, 2015
@Centril Centril added the A-macros Macro related proposals and issues label Nov 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-macros Macro related proposals and issues final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. T-lang Relevant to the language team, which will review and decide on the RFC.
Projects
None yet
Development

Successfully merging this pull request may close these issues.