Skip to content
This repository has been archived by the owner on Jun 15, 2023. It is now read-only.

Done, but do not merge yet: Async/Await syntax. #600

Merged
merged 22 commits into from
Jul 18, 2022

Conversation

IwanKaramazow
Copy link
Contributor

@IwanKaramazow IwanKaramazow commented Jul 4, 2022

Async/await surface syntax cf. https://forum.rescript-lang.org/t/ann-async-await-is-coming-to-rescript/3488

@cristianoc
Copy link
Contributor

Do we need to make await a keyword?
I was hoping we could use the sam tricks as for other non-keywords.

@IwanKaramazow
Copy link
Contributor Author

Do we need to make await a keyword?

We don't, will remove

@IwanKaramazow
Copy link
Contributor Author

IwanKaramazow commented Jul 6, 2022

I'm wondering how we should go about the implementation of await.
In JavaScript await is a keyword only inside async functions.

let await = 1 // OK ✅

async function wait() {
  let await = 1 // NOT OK ❌
}

The plot twist is that the newer browsers (chrome or node) support top-level await. (By implicitly wrapping everything in an async function?)

let await = 1  // NOT OK, with top level await ❌

This makes me think:

  • We can replicate JavaScript parser; inside async function expressions, use of await as identifier will result into a syntax error
  • Allow await as a "non-keyword" everywhere with some "tricks".
let await = 1
let x = await // Ok, now it gets tricky. Does this start an "await"-expression? 
      = await fetch(url) // could be this
      = await() // could be this
      = await await() // or even this
      = await // or we might just be assigning the identifier `await` to `x`
        

Seems like we can infer the following rules for an await expression:

  • space after await
  • next token is on the same line as await
  • next token indicates the start of a unary expression

But this "breaks down" in the case of

let x = await - 1
// could be a binary expression: "identifier" MINUS "1"
// could also be an await expression with a unary expression: `await` `(-1)`

These whitespace sensitive rules kinda feel super arbitrary. Which makes me think that introducing await as a keyword is an ok compromise.

@cristianoc
Copy link
Contributor

Indeed looks like a case where not-a-keyword costs way more than it gives.

@IwanKaramazow IwanKaramazow requested review from cristianoc and cknitt and removed request for cristianoc July 6, 2022 09:58
@IwanKaramazow IwanKaramazow marked this pull request as ready for review July 6, 2022 09:58
src/res_parsetree_viewer.mli Outdated Show resolved Hide resolved
Copy link
Contributor

@cristianoc cristianoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there roundtrip tests for this? Can't remember what tests are auto included in roundtrip.

This look great!
Only got style comments.

src/res_core.ml Outdated Show resolved Hide resolved
src/res_core.ml Show resolved Hide resolved
src/res_core.ml Show resolved Hide resolved
src/res_core.ml Outdated Show resolved Hide resolved
src/res_core.ml Outdated Show resolved Hide resolved
@@ -151,7 +151,7 @@ let isExprStart = function
| Underscore (* _ => doThings() *)
| Uident _ | Lident _ | Hash | Lparen | List | Module | Lbracket | Lbrace
| LessThan | Minus | MinusDot | Plus | PlusDot | Bang | Percent | At | If
| Switch | While | For | Assert | Lazy | Try ->
| Switch | While | For | Assert | Await | Lazy | Try ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder: in the vscode extension, the grammar needs to be taught about the keyword await.
Also, semantic highlighting needs to be taught about the id async, when encountered as attribute. The attribute's location needs to be used to emit the symbol.

src/res_grammar.ml Outdated Show resolved Hide resolved
@@ -338,6 +350,8 @@ let jsxChildExpr expr =
}
when startsWithMinus x ->
Parenthesized
| _ when ParsetreeViewer.hasAwaitAttribute expr.pexp_attributes ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of these. Should there be a generic concept of which this is an instance so all these calls are moved to a single place?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give an example of what you mean by this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can move to a helper function, though the difference is minimal. Just easier to make sure all the instances are updated at once if something changes in future.

Another thing I was wondering is: this case seems to often go where there is also ParsetreeViewer.filterParsingAttrs attrs. But not always, so not sure what value this observation has.
But I guess in the cases where ParsetreeViewer.filterParsingAttrs attrs is checked, this await check could at least go right after it. Assuming that no cases in between should take precedence.

src/res_printer.ml Outdated Show resolved Hide resolved
IwanKaramazow pushed a commit that referenced this pull request Jul 7, 2022
cristianoc added a commit to cristianoc/rescript-compiler-experiments that referenced this pull request Jul 7, 2022
@cristianoc
Copy link
Contributor

See here: cristianoc/rescript-compiler-experiments@6101661

Once this is stable we can continue and make it point to this syntax commit.

@IwanKaramazow
Copy link
Contributor Author

Are there roundtrip tests for this? Can't remember what tests are auto included in roundtrip.

Yes, printer tests are roundtripped

@cristianoc cristianoc changed the title Async/Await syntax Done, but do not merge yet: Async/Await syntax. Jul 7, 2022
src/res_core.ml Show resolved Hide resolved
@@ -338,6 +350,8 @@ let jsxChildExpr expr =
}
when startsWithMinus x ->
Parenthesized
| _ when ParsetreeViewer.hasAwaitAttribute expr.pexp_attributes ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can move to a helper function, though the difference is minimal. Just easier to make sure all the instances are updated at once if something changes in future.

Another thing I was wondering is: this case seems to often go where there is also ParsetreeViewer.filterParsingAttrs attrs. But not always, so not sure what value this observation has.
But I guess in the cases where ParsetreeViewer.filterParsingAttrs attrs is checked, this await check could at least go right after it. Assuming that no cases in between should take precedence.

@@ -11,7 +11,7 @@ let arrowType ct =
process attrsBefore (arg :: acc) typ2
| {
ptyp_desc = Ptyp_arrow ((Nolabel as lbl), typ1, typ2);
ptyp_attributes = [({txt = "bs"}, _)] as attrs;
ptyp_attributes = [({txt = "bs" | "res.async"}, _)] as attrs;
} ->
let arg = (attrs, lbl, typ1) in
process attrsBefore (arg :: acc) typ2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there a separate case for process attributes?
Why not always process attributes and if nothing of interest is found then it's a no-op.
And the following match case disappears.

src/res_parsetree_viewer.ml Outdated Show resolved Hide resolved
src/res_parsetree_viewer.ml Outdated Show resolved Hide resolved
cristianoc added a commit to cristianoc/rescript-compiler-experiments that referenced this pull request Jul 7, 2022
@cristianoc
Copy link
Contributor

Sweet. Let me sync this up to the async compiler PR.

@cristianoc
Copy link
Contributor

Bingo. Everything works now, as far as I can see.

cristianoc pushed a commit that referenced this pull request Jul 11, 2022
cristianoc added a commit to cristianoc/rescript-compiler-experiments that referenced this pull request Jul 11, 2022
cristianoc added a commit to cristianoc/rescript-compiler-experiments that referenced this pull request Jul 11, 2022
@cristianoc cristianoc added this to the v10.1 "The New Features" milestone Jul 11, 2022
@cristianoc
Copy link
Contributor

@IwanKaramazow could you rebase on master? Then we can merge.

Maxim added 4 commits July 18, 2022 09:15
operand-expr += `await` unary-expr
operand-expr += `async` es6-arrow-expression
Existing code using async/await as identifiers in e.g. record field names will keep working.
Maxim added 17 commits July 18, 2022 09:19
Reasoning:
In JavaScript `await` is a keyword *only* inside async functions.
```javascript
let await = 1 // OK ✅

async function wait() {
  let await = 1 // NOT OK ❌
}
```

The plot twist is that the newer browsers (chrome or node) support top-level await. (By implicitly wrapping everything in an async function?)

```javascript
let await = 1  // NOT OK, with top level await ❌
```

This makes me think:
* We can replicate JavaScript parser; inside async function expressions, use of `await` as identifier will result into a syntax error
  - Downside: this feels like a responsibility for another part of the compiler, not the parser? This is already implemented in cristianoc/rescript-compiler-experiments#1, so we would also be doing double work.
  - Other downside: if we ever allow top-level await, then we just implemented the above for nothing.
* Allow `await` as a "non-keyword" everywhere with some "tricks".

```javascript
let await = 1
let x = await // Ok, now it gets tricky. Does this start an "await"-expression?
      = await fetch(url) // could be this
      = await() // could be this
      = await await() // or even this
      = await // or we might just be assigning the identifier `await` to `x`

```

 Seems like we can infer the following rules for an `await expression`:
   - space after `await`
   - next token is on the same line as `await`
   - next token indicates the start of a unary expression

 But this "breaks down" in the case of
 ```javascript
 let x = await - 1
 // could be a binary expression: "identifier" MINUS "1"
 // could also be an await expression with a unary expression: `await` `(-1)`
 ```
These whitespace sensitive rules kinda feel super arbitrary. Which makes me think that introducing `await` as a keyword is an ok compromise.
@IwanKaramazow
Copy link
Contributor Author

@cristianoc done

@cristianoc cristianoc merged commit 62ccbc1 into master Jul 18, 2022
cristianoc pushed a commit that referenced this pull request Jul 18, 2022
@cristianoc cristianoc deleted the mvalcke/async-await-syntax branch July 18, 2022 08:03
cristianoc added a commit to rescript-lang/rescript that referenced this pull request Jul 18, 2022
cristianoc added a commit to rescript-lang/rescript that referenced this pull request Jul 18, 2022
cristianoc added a commit to rescript-lang/rescript that referenced this pull request Jul 18, 2022
cristianoc added a commit to rescript-lang/rescript that referenced this pull request Jul 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants