Implement Top level await #6457

rhuanjl · 2020-06-05T09:37:31Z

This PR implements the stage 3 proposal Top Level Await: https://github.com/tc39/proposal-top-level-await.

I wrote this code in December but it was stalled due to waiting for the generator/async refactor to land.

This makes the following changes:

In the parser if await is found in a module it is marked as async module.
A new reportModuleCompletionCallback is added - if set this is used for reporting the completion of all root modules, if not set, old behaviour is followed for modules without await completion of modules with await will not be reported.
Module evaluation for modules containing await now uses Async function logic for execution, a counter of executing modules (paused by await) is used to trigger parent execution at the correct time.
Some drive by cleanup in a few places.
Feature is guarded by the flag ESTopLevelAwait but this is set to enabled by default.

TODO: investigate the odd case of a module that is both a root module/main entry point AND is dynamically imported - not sure if error handling works correctly for this - also need a test for that case.

Fix: #6262

rhuanjl · 2020-06-05T09:38:37Z

bin/ch/WScriptJsrt.cpp

@@ -600,30 +600,6 @@ JsValueRef WScriptJsrt::LoadScriptHelper(JsValueRef callee, bool isConstructCall
    return returnValue;
 }

-JsErrorCode WScriptJsrt::InitializeModuleInfo(JsModuleRecord moduleRecord)


Drive by clean up in the host with past changes to module logic there was no reason for this to be a separate function any more - it's a one off initialisation step.

rhuanjl · 2020-06-05T09:39:30Z

bin/ch/WScriptJsrt.cpp

@@ -2039,6 +2005,17 @@ HRESULT WScriptJsrt::ModuleMessage::Call(LPCSTR fileName)
    return errorCode;
 }

+JsErrorCode WScriptJsrt::ReportModuleCompletionCallback(JsModuleRecord module, JsValueRef exception)


Callback used when a module completes - returns the exception if the completion was not a normal completion.

This had to be added to support asynchronous module completion - note see below if the callback is not set engine will fallback to direct reporting for modules without Top Level Await to avoid any compatability break.

rhuanjl · 2020-06-05T09:40:51Z

lib/Jsrt/Core/JsrtContextCore.cpp

+    if (reportModuleCompletionCallback == nullptr)
+    {
+        return false;
+    }


If the host has not set the callback just return false - this is to avoid backwards compatibility breaks.

rhuanjl · 2020-06-05T09:43:48Z

lib/Jsrt/Core/JsrtCore.cpp

-            moduleHostInfo != JsModuleHostInfo_FetchImportedModuleFromScriptCallback &&
-            moduleHostInfo != JsModuleHostInfo_NotifyModuleReadyCallback &&
-            moduleHostInfo != JsModuleHostInfo_InitializeImportMetaCallback)
+        if (moduleHostInfo == JsModuleHostInfo_Exception ||


Invert the condition as the number of callback types is now greater than the number of non-callbacks

rhuanjl · 2020-06-05T09:45:06Z

lib/Parser/Parse.cpp

+                if (IsTopLevelModuleFunc())
+                {
+                    MakeModuleAsync();
+                }


Convert module into an async function when await is found in it.

rhuanjl · 2020-06-05T09:47:17Z

lib/Runtime/Language/SourceTextModuleRecord.cpp

-            else
-            {
-                if (childModuleRecord->ConfirmChildrenParsed())
+        if (childrenModuleSet != nullptr)


Drive by improvement - previously in the case of no child modules an empty array was allocated here.

rhuanjl · 2020-06-05T09:51:44Z

lib/Runtime/Language/SourceTextModuleRecord.cpp

        {
-            JavascriptExceptionOperators::DoThrowCheckClone(exception, scriptContext);
+            Arguments outArgs(CallInfo(CallFlags_Value, 0), nullptr);
+            this->generator = VarTo<JavascriptGenerator>(rootFunction->CallRootFunction(outArgs, scriptContext, true));


This call executes the "header" of the module which hoists any function exports. It yields before the body of the module.

rhuanjl · 2020-06-05T09:52:54Z

lib/Runtime/Language/SourceTextModuleRecord.cpp

+            else
+            {
+                SetWasEvaluated();
+                SetEvaluating(false);
+                gen->CallGenerator(scriptContext->GetLibrary()->GetUndefined(), ResumeYieldKind::Normal);
+                if (this->parentModuleList != nullptr && !shouldIncrementAwait)
+                {
+                    parentModuleList->Map([=](uint i, SourceTextModuleRecord* parentModule)
+                    {
+                        if (parentModule->DecrementAwaited())
+                        {
+                            parentModule->FinishModuleEvaluation(false);
+                        }
+                    });
+                }


Module without top level await - execute the whole thing.

rhuanjl · 2020-06-05T09:53:41Z

lib/Runtime/Language/SourceTextModuleRecord.cpp

+            if (gen->IsAsyncModule())
+            {
+                JavascriptPromise* prom = JavascriptAsyncFunction::BeginAsyncFunctionExecution(this->generator);
+                if (this->parentModuleList != nullptr && shouldIncrementAwait)
+                {
+                    parentModuleList->Map([=](uint i, SourceTextModuleRecord* parentModule)
+                    {
+                        parentModule->IncrementAwaited();
+                    });
+                }
+                auto* fulfilled = scriptContext->GetLibrary()->CreateAsyncModuleCallbackFunction(EntryAsyncModuleFulfilled, this);
+                auto* rejected = scriptContext->GetLibrary()->CreateAsyncModuleCallbackFunction(EntryAsyncModuleRejected, this);
+                auto* unused = JavascriptPromise::UnusedPromiseCapability(scriptContext);
+                JavascriptPromise::PerformPromiseThen(prom, unused, fulfilled, rejected, scriptContext);


Module with top level await execute until the first await then PerformPromiseThen to queue the continuation.

rhuanjl · 2020-06-05T09:55:16Z

lib/Runtime/Library/JavascriptAsyncFunction.cpp

+    return BeginAsyncFunctionExecution(generator);
+}
+
+JavascriptPromise* JavascriptAsyncFunction::BeginAsyncFunctionExecution(JavascriptGenerator* generator)
+{
+    auto* library = generator->GetLibrary();


Async function implementation split in two - as second half which actually executes the function can be reused for modules with Top level Await, whilst first half is not needed for modules.

rhuanjl · 2020-06-05T10:17:54Z

test/es6module/bug_OS12095746.baseline

-NotifyModuleReadyCallback(exception) bug_OS12095746_mod0.js
-NotifyModuleReadyCallback(exception) bug_OS12095746_mod1.js
 mod0 catch:Syntax error
 mod1 catch:Unexpected identifier after numeric literal
-NotifyModuleReadyCallback(exception) bug_OS12095746_mod2.js


This callback no longer fires for Dynamic module exceptions - there was no useful purpose to it firing before as Host ought to disregard it and leaving handling to within script - as promise is rejected.

rhuanjl · 2020-06-05T10:26:46Z

@ppenzin I'm aware that this isn't simple but would be great if you could take a look

@chicoxyzzy would be good if you could comment on the tests I've added for this - this wasn't the easiest spec to implement in light of both its complexity and the existing ChakraCore logic in these areas - so want to be sure the tests are hitting the key paths.

rhuanjl · 2020-12-05T21:31:42Z

I've gone over this again and rebased it - I think it's solid; aware it's not trivial but would be great if someone could take a look, @Fly-Style @ppenzin

ppenzin · 2020-12-08T02:00:57Z

I've started looking at this a couple of times, but got sidetracked by easier things. Will take another look.

lib/Runtime/Library/JavascriptGeneratorFunction.cpp

rhuanjl commented Jun 5, 2020

View reviewed changes

rhuanjl force-pushed the TopLevelAwait branch from 5abbb2f to 9fa084f Compare December 5, 2020 21:21

Implement Top Level Await

8c3a82b

rhuanjl force-pushed the TopLevelAwait branch from 9fa084f to 8c3a82b Compare December 6, 2020 00:16

ppenzin reviewed Dec 18, 2020

View reviewed changes

lib/Runtime/Library/JavascriptGeneratorFunction.cpp Show resolved Hide resolved

ppenzin approved these changes Dec 18, 2020

View reviewed changes

lib/Runtime/Library/JavascriptGeneratorFunction.cpp Show resolved Hide resolved

ppenzin merged commit 9e2f198 into chakra-core:master Dec 18, 2020

rhuanjl deleted the TopLevelAwait branch December 18, 2020 15:46

rhuanjl mentioned this pull request Jan 26, 2021

Advance to stage 4 tc39/proposal-top-level-await#113

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Top level await #6457

Implement Top level await #6457

rhuanjl commented Jun 5, 2020 •

edited

Loading

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl Jun 5, 2020

rhuanjl commented Jun 5, 2020

rhuanjl commented Dec 5, 2020

ppenzin commented Dec 8, 2020

Implement Top level await #6457

Implement Top level await #6457

Conversation

rhuanjl commented Jun 5, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhuanjl commented Jun 5, 2020

rhuanjl commented Dec 5, 2020

ppenzin commented Dec 8, 2020

rhuanjl commented Jun 5, 2020 •

edited

Loading