Refactor parsing of trait object types #40043

petrochenkov · 2017-02-23T00:13:47Z

Bugs are fixed and code is cleaned up.

User visible changes:

ty matcher in macros accepts trait object types like Write + Send (macros: ty matcher doesn't accept trait object types with + (type sums) #39080)
Buggy priority of + in trait object types starting with for is fixed (syntax: + has incorrect priority in trait object types starting with for #39317). &for<'a> Trait<'a> + Send is now parsed as (&for<'a> Trait<'a>) + Send and requires parens &(for<'a> Trait<'a> + Send). For comparison, &Send + for<'a> Trait<'a> was parsed like this since Nov 27, 2014.
Trailing +s are supported in trait objects, like in other bounds.
Better error reporting for trait objects starting with ?Sized.

Fixes #39080
Fixes #39317 [breaking-change]
Closes #39298
cc #39085 (fixed, then reverted #40043 (comment))
cc #39318 (fixed, then reverted #40043 (comment))

r? @nikomatsakis

bors · 2017-03-01T07:59:48Z

☔ The latest upstream changes (presumably #39419) made this pull request unmergeable. Please resolve the merge conflicts.

petrochenkov · 2017-03-01T11:06:20Z

Rebased.

nikomatsakis · 2017-03-01T23:39:06Z

@petrochenkov these changes look good to me. Sorry for delay. Will queue up a crater run.

nikomatsakis · 2017-03-03T21:36:26Z

Regression report: https://gist.github.com/nikomatsakis/42c2e8f0ff770878fed845d93c53c05e

c_vec-1.2.0 (before) (after) -- timeout
jmespath-0.1.1 (before) (after) -- Box<(Fn(&[Rcvar], &mut Context) -> SearchResult) + Sync>, it feels like this should work
cat-0.0.1 (before) (after) -- timeout
incrust-0.2.12 (before) (after) -- & for <'t> Type<'t> + 'r -- I think this is a bug you intended to fix?
plex-0.0.3 (before) (after) -- "no method named eat_lifetime found for type syntax::parse::parser::Parser<'_> in the current scope", use of unstable API
schedule-0.1.0 (before) (after) -- (FnMut() -> ()) + 'a, also seems like this should work
shadowsocks-0.6.1 (before) (after) -- Box<(Fn(D::Error) -> E) + 'static+ Send+Sync>, seems like this should work
throw-0.1.1 (before) (after) -- timeout

So, I think we should fix the parenthesis thing. Not sure what to do about the precedence of for<'a> vis-a-vis + exactly. We might just submit a patch and try to issue a new point release (cc @alexander-irbis -- your crate incrust may have been relying on a parsing bug).

petrochenkov · 2017-03-03T22:05:19Z

@nikomatsakis
jmespath-0.1.1, schedule-0.1.0 and shadowsocks-0.6.1 are #39318 and I don't think it should work because the fact that it works now is an accidental regression and parenthesized bounds are never accepted in other situations. I'll add code parsing these parentheses, but it needs at least a warning.

We might just submit a patch

Yeah, I'll send patches to affected crates.

petrochenkov · 2017-03-05T00:15:41Z

Updated with a warning instead of an error for (Trait) + Trait + 'lt.
Patches to affected crates are submitted.

nikomatsakis · 2017-03-06T18:53:19Z

@petrochenkov I'm not sure I agree that () should not work. It may have been an accidental regression, but it like what I would naively expect. That is, + is a type that combines other things that fit the type grammar (which should be object-safe traits, essentially). It has lower precedence than prefix operators like &, hence we write &(Foo + Safe), as well as the fn "operator" (hence fn foo() -> (Foo + Safe)). But I see no reason that more parens can't work (e.g., &((Foo) + Safe)).

petrochenkov · 2017-03-06T22:09:55Z

@nikomatsakis

That is, + is a type that combines other things that fit the type grammar

But... that's not true (the lifetime in Trait + 'a already doesn't fit into the type grammar). "+" is a list of generic bounds, exactly the same thing as in fn f<T: HERE>() {} or fn f() -> impl HERE and it should reuse the same grammar, for simplicity and clarity.
The current scheme treats the first bound specially and inconsistently with other bounds, my goal was to remove this special case. In theory parentheses may be permitted in all bounds (fn f<T: (Debug) + (Fn(u8) -> u8) + ('static)>, but it would be such a useless addition to the language, that I'd rather prefer to prohibit the existing special case instead. (This prohibition is also kind of an unused_parens lint that is always enabled (with a nice error message suggesting to remove the parens!))

nikomatsakis · 2017-03-07T22:57:17Z

src/libsyntax/parse/parser.rs

+                let ty = ts.into_iter().nth(0).unwrap().unwrap();
+                match ty.node {
+                    // Accept `(Trait1) + Trait2 + 'a` for now,
+                    // but issue a future compatibility warning.


in what sense is this a "future-compatiblity warning"?

specifically, it doesn't seem to mention anything about the future? (and there is no associated tracking issue, right?)

I repurposed #39318 as a tracking issue, I'll link to it.

nikomatsakis · 2017-03-07T23:00:55Z

@petrochenkov Hmm, first, it's not clear that 'a should be considered specially -- it may be reasonable to consider it as an "object type", just like any other. I also feel like I'd expect parens to be legal in a list of bounds, for that matter. But I'm not sure how much I care, ultimately, I'd probabbly be ok with the conservative route for now as long we get the crates updated.

cc @rust-lang/lang -- some questions here about the finer points of our grammar

petrochenkov · 2017-03-07T23:25:52Z

@nikomatsakis

Hmm, first, it's not clear that 'a should be considered specially -- it may be reasonable to consider it as an "object type", just like any other.

That's what I originally did, but then added 63c5c0d just to be conservative and not accept a single lifetime in ty matcher, who knows what problems this could cause in the future.
I'm not really sure what is the right thing to do here and will gladly accept any consensus the lang team will come up with.

withoutboats · 2017-03-07T23:47:52Z

That's what I originally did, but then added 63c5c0d just to be conservative and not accept a single lifetime in ty matcher, who knows what problems this could cause in the future.

I can't think of any problems, but it feels like we've already made our bed here.

As useless as this is, philosophically it seems like a type like &'a 'b should be accepted.

This becomes more material in light of rust-lang/rfcs#1733, which would presumably allow these through an alias even if they're ungrammatical directly. That is, this would compile:

trait Static = 'static;
fn foo(_: Box<Static>) { }

So why wouldn't this?

fn foo(_: Box<'static>) { }

nikomatsakis · 2017-03-08T17:15:21Z

Sigh, I hadn't considered the parsing ambiguity here:

As useless as this is, philosophically it seems like a type like &'a 'b should be accepted.

But it can be resolved easily enough via some kind of rules that require an explicit paren, I guess, or something.

nikomatsakis · 2017-03-08T17:16:03Z

This becomes more material in light of rust-lang/rfcs#1733, which would presumably allow these through an alias even if they're ungrammatical directly. That is, this would compile:

Well, this could be considered not object safe, if we had a rule that there must be at least 1 actionable trait or something.

withoutboats · 2017-03-08T18:27:05Z

Well, this could be considered not object safe, if we had a rule that there must be at least 1 actionable trait or something.

That's true. I'm okay with that if parsing these types is too complex.

nikomatsakis · 2017-03-09T21:34:50Z

In @rust-lang/compiler meeting, we decided not to beta-backport this: it's a big change, and it doesn't really seem to fix anything tagged as regressions, except for #39318 -- and we're still debating what's the desired behavior here (and it would have a warning period anyhow).

petrochenkov · 2017-03-16T22:04:13Z

VVV
@nikomatsakis
Updated, the warning for parens is removed, Box<'a + Trait> is un-implemented.
/\/\/\

Nevertheless, this is all very strange

Why would we want to support Box<'a + Bounds>? It seems to me that this is actively undesirable. (In particular, because it seems to suggest that 'a by itself would be allowed, and we don't want that, and because it means that Box<'a> vs Box<'a+> seem quite different...?)

I guess what I mean is, if the lifetime bound can't be the only thing that is present, why allow it to go first?

impl 'a + Trait is supported => Box<'a + Trait> should be supported
'a has syntactic ambiguity, 'a + whatever doesn't have syntactic ambiguity and can be supported.
I don't see any link between 'a + Trait being allowed and and 'a being allowed.
I don't see any link between 'a being allowed or disallowed and 'a going first.
I don't see why the first bound need to be treated specially at all.

solson · 2017-03-16T22:13:14Z

impl 'a + Trait is supported

To be honest, I find this surprising, too.

I don't see why the first bound need to be treated specially at all.

I read a trait object as MainTrait + extra where extra can include only a small selection of marker traits and a lifetime. I can see the value of relaxing the parsing rules, I suppose, since one might not expect a strict ordering around +, but when it's supported I will consider Box<Send + Trait> and Box<'a + Trait> examples of bad style.

petrochenkov · 2017-03-16T22:46:46Z

@solson

I read a trait object as MainTrait + extra where extra can include only a small selection of marker traits and a lifetime.

This is a model dictated by limitations of the current implementation - it is quite reasonable to expect Box<Write + Read + Debug> to work, and to point to something implementing all these traits (in any order), but there you have implementation issues like how to combine vtables, etc.
But foremost, this all is not a parser's concern (rejecting legal but dubious style is not a parser's concern either, IMO).

solson · 2017-03-16T23:07:59Z

@petrochenkov Multiple trait object support would definitely change how I think about this, although I would recommend always putting the traits with methods first in the list.

But foremost, this all is not a parser's concern (rejecting legal but dubious style is not a parser's concern either, IMO).

Yeah, that's fair. I do see the sense in relaxing the parsing rules and/or making them more consistent. I'm basically just saying the restriction isn't very strange from a user point-of-view.

nikomatsakis · 2017-03-20T18:58:14Z

@petrochenkov

Well, I guess my sense is that if A + B parses, I expect A to parse, and not have a radically different meaning. But I agree there is no technical objection per se. (And, to be fair, I also expect B + A to parse, so... there is no winning. =)

I suppose that if we adopted the dyn Trait proposal, then both impl and dyn unambiguously take a list of bounds afterwards, and all potential confusion is removed (modulo questions of precedence. Perhaps a point in favor.

withoutboats · 2017-03-20T19:08:36Z

So I've just discovered that this is valid:

fn foo<'a, T: 'a + ToString>() { }

I think trait object syntax should be consistent with bounds syntax, so Box<'a + ToString> should be valid. Maybe if we decide to lint more stylistic things we should lint a preference to lifetime constraints coming last.

nikomatsakis · 2017-03-20T19:40:45Z

@bors r+

bors · 2017-03-20T19:40:47Z

📌 Commit 803e76c has been approved by nikomatsakis

bors · 2017-03-21T19:34:53Z

🔒 Merge conflict

petrochenkov · 2017-03-21T20:54:27Z

@bors r=nikomatsakis

bors · 2017-03-21T20:54:28Z

📌 Commit b5e8897 has been approved by nikomatsakis

bors · 2017-03-22T02:00:20Z

⌛ Testing commit b5e8897 with merge 8c4f2c6...

@nikomatsakis

Refactor parsing of trait object types Bugs are fixed and code is cleaned up. User visible changes: - `ty` matcher in macros accepts trait object types like `Write + Send` (#39080) - Buggy priority of `+` in trait object types starting with `for` is fixed (#39317). `&for<'a> Trait<'a> + Send` is now parsed as `(&for<'a> Trait<'a>) + Send` and requires parens `&(for<'a> Trait<'a> + Send)`. For comparison, `&Send + for<'a> Trait<'a>` was parsed like this since [Nov 27, 2014](#19298). - Trailing `+`s are supported in trait objects, like in other bounds. - Better error reporting for trait objects starting with `?Sized`. Fixes #39080 Fixes #39317 [breaking-change] Closes #39298 cc #39085 (fixed, then reverted #40043 (comment)) cc #39318 (fixed, then reverted #40043 (comment)) r? @nikomatsakis

bors · 2017-03-22T04:35:50Z

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing 8c4f2c6 to master...

The merge of rust-lang/rust#40043 removes parse_ty_path in the latest nightly, which we depended on. This patch rewrites that code path using parse_path, and in the process eliminates an unreachable!() if let arm.

syntax: Parse trait object types starting with a lifetime bound Fixes #39085 This was originally implemented in #40043, then reverted, then there was some [agreement](#39318 (comment)) that it should be supported. (This is hopefully the last PR related to bound parsing.)

rust-highfive assigned nikomatsakis Feb 23, 2017

petrochenkov added the S-waiting-on-crater Status: Waiting on a crater run to be completed. label Feb 23, 2017

petrochenkov force-pushed the objpars branch from e783e10 to 9af340b Compare March 1, 2017 11:06

petrochenkov force-pushed the objpars branch from 9af340b to 6869cf8 Compare March 4, 2017 20:40

This was referenced Mar 4, 2017

Fix syntactic error on nightly Rust jmespath/jmespath.rs#20

Merged

Fix syntactic error on nightly Rust mehcode/schedule-rs#14

Merged

Fix syntactic error on nightly Rust irbis-labs/incrust#2

Merged

petrochenkov force-pushed the objpars branch from 6869cf8 to 8e7803f Compare March 5, 2017 00:13

petrochenkov removed the S-waiting-on-crater Status: Waiting on a crater run to be completed. label Mar 5, 2017

pmarcelll mentioned this pull request Mar 5, 2017

Parsing issues with Box'ed unboxed closures #20640

Closed

petrochenkov mentioned this pull request Mar 5, 2017

syntax: Trait object types permit extra parentheses on the first bound #39318

Closed

2 tasks

petrochenkov force-pushed the objpars branch from 8e7803f to 63c5c0d Compare March 5, 2017 21:53

nikomatsakis reviewed Mar 7, 2017

View reviewed changes

brson added beta-nominated Nominated for backporting to the compiler in the beta channel. relnotes Marks issues that should be documented in the release notes of the next release. labels Mar 9, 2017

petrochenkov force-pushed the objpars branch from 63c5c0d to 803e76c Compare March 16, 2017 21:49

Refactor parsing of trait object types

b5e8897

retep998 force-pushed the objpars branch from 803e76c to eff97c7 Compare March 21, 2017 20:04

petrochenkov force-pushed the objpars branch from eff97c7 to b5e8897 Compare March 21, 2017 20:54

bors mentioned this pull request Mar 22, 2017

appveyor: Use Ninja to build LLVM on MinGW #40548

Merged

bors merged commit b5e8897 into rust-lang:master Mar 22, 2017

This was referenced Mar 22, 2017

macros: improve Span's expansion information #40597

Merged

Add diagnostic for incorrect pub (restriction) #40627

Merged

petrochenkov mentioned this pull request Mar 22, 2017

Can't compile type with lifetime bound in certain order #39085

Closed

jonhoo mentioned this pull request Mar 23, 2017

Fix following removal of parse_ty_path google/tarpc#138

Merged

petrochenkov mentioned this pull request Apr 25, 2017

syntax: Parse trait object types starting with a lifetime bound #41542

Merged

petrochenkov deleted the objpars branch August 26, 2017 00:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor parsing of trait object types #40043

Refactor parsing of trait object types #40043

petrochenkov commented Feb 23, 2017 •

edited

Loading

bors commented Mar 1, 2017

petrochenkov commented Mar 1, 2017

nikomatsakis commented Mar 1, 2017

nikomatsakis commented Mar 3, 2017 •

edited

Loading

petrochenkov commented Mar 3, 2017

petrochenkov commented Mar 5, 2017

nikomatsakis commented Mar 6, 2017

petrochenkov commented Mar 6, 2017 •

edited

Loading

nikomatsakis Mar 7, 2017

nikomatsakis Mar 7, 2017

petrochenkov Mar 7, 2017

nikomatsakis commented Mar 7, 2017

petrochenkov commented Mar 7, 2017 •

edited

Loading

withoutboats commented Mar 7, 2017

nikomatsakis commented Mar 8, 2017

nikomatsakis commented Mar 8, 2017

withoutboats commented Mar 8, 2017

nikomatsakis commented Mar 9, 2017 •

edited

Loading

petrochenkov commented Mar 16, 2017 •

edited

Loading

solson commented Mar 16, 2017 •

edited

Loading

petrochenkov commented Mar 16, 2017

solson commented Mar 16, 2017

nikomatsakis commented Mar 20, 2017 •

edited

Loading

withoutboats commented Mar 20, 2017

nikomatsakis commented Mar 20, 2017

bors commented Mar 20, 2017

bors commented Mar 21, 2017

petrochenkov commented Mar 21, 2017

bors commented Mar 21, 2017

bors commented Mar 22, 2017

bors commented Mar 22, 2017

Refactor parsing of trait object types #40043

Refactor parsing of trait object types #40043

Conversation

petrochenkov commented Feb 23, 2017 • edited Loading

bors commented Mar 1, 2017

petrochenkov commented Mar 1, 2017

nikomatsakis commented Mar 1, 2017

nikomatsakis commented Mar 3, 2017 • edited Loading

petrochenkov commented Mar 3, 2017

petrochenkov commented Mar 5, 2017

nikomatsakis commented Mar 6, 2017

petrochenkov commented Mar 6, 2017 • edited Loading

nikomatsakis Mar 7, 2017

Choose a reason for hiding this comment

nikomatsakis Mar 7, 2017

Choose a reason for hiding this comment

petrochenkov Mar 7, 2017

Choose a reason for hiding this comment

nikomatsakis commented Mar 7, 2017

petrochenkov commented Mar 7, 2017 • edited Loading

withoutboats commented Mar 7, 2017

nikomatsakis commented Mar 8, 2017

nikomatsakis commented Mar 8, 2017

withoutboats commented Mar 8, 2017

nikomatsakis commented Mar 9, 2017 • edited Loading

petrochenkov commented Mar 16, 2017 • edited Loading

solson commented Mar 16, 2017 • edited Loading

petrochenkov commented Mar 16, 2017

solson commented Mar 16, 2017

nikomatsakis commented Mar 20, 2017 • edited Loading

withoutboats commented Mar 20, 2017

nikomatsakis commented Mar 20, 2017

bors commented Mar 20, 2017

bors commented Mar 21, 2017

petrochenkov commented Mar 21, 2017

bors commented Mar 21, 2017

bors commented Mar 22, 2017

bors commented Mar 22, 2017

petrochenkov commented Feb 23, 2017 •

edited

Loading

nikomatsakis commented Mar 3, 2017 •

edited

Loading

petrochenkov commented Mar 6, 2017 •

edited

Loading

petrochenkov commented Mar 7, 2017 •

edited

Loading

nikomatsakis commented Mar 9, 2017 •

edited

Loading

petrochenkov commented Mar 16, 2017 •

edited

Loading

solson commented Mar 16, 2017 •

edited

Loading

nikomatsakis commented Mar 20, 2017 •

edited

Loading