Versions, editions, features

[ehuss]

• How will the grammar track the rustc version?
• How will the grammar handle editions?
• Will the grammar support unstable features/syntax?
• How to handle/highlight deprecated syntax?

[CAD97]

I think that the initial target should be "stable 1.31 edition 2018" or similar. We shouldn't worry about unstable parts of the grammar or edition switches for the first pass. (I think this was mentioned and suggested without argument in the first meeting.)

[Centril]

From today's meeting: By "1.31" we mean the latest stable, not literally 1.31.

[Centril]

Also from today's meeting: It is too early to think about unstable features and such.

[eddyb]

Quick thought: if we add rules like this:

EDITION_2015 = ;
EDITION_2018 = ;

FEAT_CONST_GENERICS = ;
// etc.

Then we can have (assuming rust-lang/gll#141 and KEYWORD containing "dyn"):

Ident =
  | IDENT
  | EDITION_2015 "dyn"
  // ...
  ;

FnHeader = constness:"const"? asyncness:{ EDITION_2018 "async" }? unsafety:"unsafe"? { "extern" abi:Abi }?;

GenericParamKind =
  // ...
  | Const:{ FEAT_CONST_GENERICS "const" name:IDENT ":" ty:Type }
  ;

They will always succeed without consuming tokens, but they'll still be in resulting parse forest.

So something neat we can then do, for a given parse result is find these to determine which editions/features a given input depends on. And we get Spans, so, feature-gating!

But also, if both EDITION_2015 and EDITION_2018 are needed in the same input (e.g. using dyn as a regular identifier and using async fn), then that's an error.

We could still treat treat that as a "parameterized grammar" by making some of the rules never succeed instead, and with the grammer interpreter that'd be at runtime anyway, so we could e.g. have --edition 2018 style flags.

[matklad]

But also, if both EDITION_2015 and EDITION_2018 are needed in the same input (e.g. using dyn as a regular identifier and using async fn), then that's an error.

Is this actually true? My understanding is that we track editions on per-token basis, and that a given snippet of code might contain a mixture of tokens from different editions because of macros.

[eddyb]

@matklad Sure, but that's not something we can test in this repo, I was talking about the expansion-less situation.

Still, we have access to the Spans so theoretically, if we were enforcing instead of just scanning the parse forest, we could apply the expected behavior. I guess this would be useful for disambiguation.

[Centril]

cc @petrochenkov