Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Async Functions #1664

Closed
rbuckton opened this issue Jan 14, 2015 · 188 comments
Closed

Proposal: Async Functions #1664

rbuckton opened this issue Jan 14, 2015 · 188 comments
Assignees
Labels
Committed The team has roadmapped this issue ES Next New featurers for ECMAScript (a.k.a. ESNext) Fixed A PR has been merged for this issue Suggestion An idea for TypeScript

Comments

@rbuckton
Copy link
Member

Async Functions

1 Async Functions

This is a spec proposal for the addition of Async Functions (also known as async..await) as a feature of TypeScript.

2 Use Cases

Async Functions allow TypeScript developers to author functions that are expected to invoke an asynchronous operation and await its result without blocking normal execution of the program. This accomplished through the use of an ES6-compatible Promise implementation, and transposition of the function body into a compatible form to resume execution when the awaited asynchronous operation completes.

This is based primarily on the Async Functions strawman proposal for ECMAScript, and C# 5.0 § 10.15 Async Functions.

3 Introduction

3.1 Syntax

An Async Function is a JavaScript Function, Parameterized Arrow Function, Method, or Get Accessor that has been prefixed with the async modifier. This modifier informs the compiler that function body transposition is required, and that the keyword await should be treated as a unary expression instead of an identifier. An Async Function must provide a return type annotation that points to a compatible Promise type. Return type inference can only be used if there is a globally defined, compatible Promise type.

Example:

var p: Promise<number> = /* ... */;  
async function fn(): Promise<number> {  
  var i = await p; // suspend execution until 'p' is settled. 'i' has type "number"  
  return 1 + i;  
}  

var a = async (): Promise<number> => 1 + await p; // suspends execution.  
var a = async () => 1 + await p; // suspends execution. return type is inferred as "Promise<number>" when compiling with --target ES6  
var fe = async function(): Promise<number> {  
  var i = await p; // suspend execution until 'p' is settled. 'i' has type "number"  
  return 1 + i;  
}  

class C {  
  async m(): Promise<number> {  
    var i = await p; // suspend execution until 'p' is settled. 'i' has type "number"  
    return 1 + i;  
  }  

  async get p(): Promise<number> {  
    var i = await p; // suspend execution until 'p' is settled. 'i' has type "number"  
    return 1 + i;  
  }  
}

3.2 Transformations

To support this feature, the compiler needs to make certain transformations to the function body of an Async Function. The type of transformations performed depends on whether the current compilation target is ES6, or ES5/ES3.

3.2.1 ES6 Transformations

During compilation of an Async Function when targeting ES6, the following transformations are applied:

  • The new function body consists of a single return statement whose expression is a new instance of the promise type supplied as the return type of the Async Function.
  • The original function body is enclosed in a Generator Function.
  • Any await expressions inside the original function body are transformed into a compatible yield expression.
    • As the precedence of yield is much lower than await, it may be necessary to enclose the yield expression in parenthesis if it is contained in the left-hand-side of a binary expression.
  • The Generator Function is executed and the resulting generator is passed as an argument to the __awaiter helper function.
  • The result of the __awaiter helper function is passed as an argument to the promise resolve callback.

Example:

// async.ts  
var p0: Promise<number> = /* ... */;  
var p1: Promise<number> = /* ... */;  
async function fn() {  
  var i = await p0;  
  return await p1 + i;  
}  

// async.js  
var __awaiter = /* ... */;  

function fn() {  
  return new Promise(function (_resolve) {  
    _resolve(__awaiter(function* () {  
      var i = yield p0;  
      return (yield p1) + i;  
    }()));  
  });  
}

3.2.2 ES5/ES3 Transformations

As ES5 and earlier do not support Generator Functions, a more complex transformation is required. To support this transformation, __generator helper function will be also emitted along with the __awaiter helper function, and a much more comprehensive set of transformations would be applied:

  • The new function body consists of a single return statement whose expression is a new instance of the promise type supplied as the return type of the Async Function.
  • The original function body is enclosed in a function expression with a single parameter that is passed as an argument to the __generator helper function.
  • All hoisted declarations (variable declarations and function declarations) in the original function body are extracted and added to the top of the new function body.
  • The original function body is rewritten into a series of case clauses in a switch statement.
  • Each statement in the function body that contains an await expression is rewritten into flattened set of instructions that are interpreted by the __generator helper function.
  • Temporary locals are generated to hold onto the values for partially-applied expressions to preserve side effects expected in the original source.
  • Logical binary expressions that contain an await expression are rewritten to preserve shortcutting.
  • Assignment expressions that contain an await are rewritten to store portions left-hand side of the assignment in temporary locals to preserve side effects.
  • Call expressions that contain an await in the argument list are rewritten to store the callee and this argument, and to instead call the call method of the callee.
  • New expressions that contain an await in the argument list are rewritten to preserve side effects.
  • Array literal expressions that contain an await in the element list are rewritten to preserve side effects.
  • Object literal expressions that contain an await in the element list are rewritten to preserve side effects.
  • try statements that contain an await in the try block, catch clause, or finally block are rewritten and tracked as a protected region by the __generator helper function.
  • The variable of a catch clause is renamed to a unique identifier and all instances of the symbol are renamed to preserve the block scope behavior of a catch variable.
  • break and continue statements whose target is a statement that contains an await expression are rewritten to return an instruction interpreted by the __generator helper function.
  • return statements are rewritten to return an instruction interpreted by the __generator helper function.
  • for..in statements that contain an await expression are rewritten to capture the enumerable keys of the expression into a temporary array, to allow the iteration to be resumed when control is returned to the function following an await expression.
  • Labeled statements that contain an await expression are rewritten.
  • await expressions are rewritten to return an instruction interpreted by the __generator helper function.

Example:

// async.ts  
var p0: Promise<number> = /* ... */;  
var p1: Promise<number> = /* ... */;  
async function fn() {  
    var i = await p0;  
    return await p1 + i;  
}  

// async.js  
var __awaiter = /* ... */;  
var __generator = /* ... */;  

function fn() {  
    var i;  
    return new Promise(function (_resolve) {  
        resolve(__awaiter(__generator(function (_state) {  
            switch (_state.label) {  
                case 0: return [3 /*yield*/, p0];  
                case 1:  
                    i = _state.sent;  
                    return [3 /*yield*/, p1];  
                case 2:  
                    return [2 /*return*/, _state.sent + i];  
            }  
        }));  
    });  
}

The following is an example of an async function that contains a try statement:

// async.ts  
var p0: Promise<number> = /* ... */;  
var p1: Promise<number> = /* ... */;  
async function fn() {  
    try {  
        await p0;  
    }  
    catch (e) {  
        alert(e.message);  
    }  
    finally {  
        await p1;  
    }  
}  

// async.js  
var __awaiter = /* ... */;  
var __generator = /* .. */;  

function fn() {  
    var i;  
    return new Promise(function (_resolve) {  
        resolve(__awaiter(__generator(function (_state) {  
            switch (_state.label) {  
                case 0:  
                    _state.trys = [];  
                    _state.label = 1;  
                case 1:  
                    _state.trys.push([1, 3, 4, 6]);  
                    return [3 /*yield*/, p0];  
                case 2:  
                    return [5 /*break*/, 6];  
                case 3:  
                    _a = _state.error;  
                    alert(_a.message);  
                    return [5 /*break*/, 6];  
                case 4:  
                    return [3 /*yield*/, p1];  
                case 5:  
                    return [6 /*endfinally*/];  
                case 6:  
                    return [2 /*return*/];  
            }  
        }));  
    });  
    var _a;  
}

As a result of these transformations, the JavaScript output for an Async Function can look quite different than the original source. When debugging the JavaScript output for an Async Function it would be advisable to use a Source Map generated using the --sourceMap option for the compiler.

4 Promise

Async Functions require a compatible Promise abstraction to operate properly. A compatible implementation implements the following interfaces, which are to be added to the core library declarations (lib.d.ts):

interface IPromiseConstructor<T> {  
    new (init: (resolve: (value: T | IPromise<T>) => void, reject: (reason: any) => void) => void): IPromise<T>;  
}  

interface IPromise<T> {  
    then<TResult>(onfulfilled: (value: T) => TResult | IPromise<TResult>, onrejected: (reason: any) => TResult | IPromise<TResult>): IPromise<TResult>;  
}

The following libraries contain compatible Promise implementations:

A Grammar

A.1 Types

  CallSignature [Await,AsyncParameter] :
   TypeParameters
opt( ParameterList [?Await,?AsyncParameter]opt) TypeAnnotation opt*

  ParameterList_ [Await,AsyncParameter] *:_
   RequiredParameterList [?Await]
   OptionalParameterList [?Await,?AsyncParameter]
   RestParameter [?Await]
   RequiredParameterList [?Await],OptionalParameterList [?Await,?AsyncParameter]
   RequiredParameterList [?Await],RestParameter [?Await]
   OptionalParameterList [?Await,?AsyncParameter],RestParameter [?Await]
   RequiredParameterList [?Await],OptionalParameterList [?Await,?AsyncParameter],RestParameter [?Await]

  RequiredParameterList [Await] :
   RequiredParameter [?Await]
   RequiredParameterList [?Await],RequiredParameter [?Await]

  RequiredParameter [Await] :
   AccessibilityModifier optBindingIdentifier [?Await]TypeAnnotation opt
   Identifier:StringLiteral

  OptionalParameterList [Await,AsyncParameter] :
   OptionalParameter [?Await,?AsyncParameter]
   OptionalParameterList [?Await,?AsyncParameter],OptionalParameter [?Await,?AsyncParameter]

  OptionalParameter [Await,AsyncParameter]:
   [+AsyncParameter] AccessibilityModifier optBindingIdentifier [Await]?TypeAnnotation opt
   [+AsyncParameter] AccessibilityModifier optBindingIdentifier [Await]TypeAnnotation optInitialiser [In]
   [+AsyncParameter] BindingIdentifier [Await]?:StringLiteral
   [~AsyncParameter] AccessibilityModifier optBindingIdentifier [?Await]?TypeAnnotation opt
   [~AsyncParameter] AccessibilityModifier optBindingIdentifier [?Await]TypeAnnotation optInitialiser [In,?Await]
   [~AsyncParameter] BindingIdentifier [?Await]?:StringLiteral

  RestParameter [Await]:
   ...BindingIdentifier [?Await]TypeAnnotation opt

A.2 Expressions

  BindingIdentifier [Await] : ( Modified )
   Identifier but not await
   [~Await] await

  PropertyAssignment [Await] :
   PropertyName:AssignmentExpression [?Await]
   PropertyNameCallSignature{FunctionBody}
   GetAccessor
   SetAccessor
   async [no LineTerminator here] PropertyNameCallSignature [Await,AsyncParameter]{FunctionBody [Await]}

  GetAccessor :
   getPropertyName()TypeAnnotationopt{FunctionBody}
   async [no LineTerminator here] getPropertyName()TypeAnnotation opt{FunctionBody [Await]}

  FunctionExpression [Await] : ( Modified )
   functionBindingIdentifier [?Await]optCallSignature{FunctionBody}
   async [no LineTerminator here] functionBindingIdentifier [?Await]optCallSignature [Await,AsyncParameter]{FunctionBody [Await]}

  AssignmentExpression [Await] : ( Modified )
   ...
   ArrowFunctionExpression [?Await]

  ArrowFunctionExpression [Await] :
   ArrowFormalParameters=>Block [?Await]
   ArrowFormalParameters=>AssignmentExpression [?Await]
   async [no LineTerminator here] CallSignature [Await,AsyncParameter]=>Block [Await]
   async [no LineTerminator here] CallSignature [Await,AsyncParameter]=>AssignmentExpression [Await]

  ArrowFormalParameters [Await] :
   CallSignature
   BindingIdentifier [?Await]

  UnaryExpression [Await] : ( Modified )
   ...
   <Type>UnaryExpression [?Await]
   [+Await] awaitUnaryExpression [Await]

A.3 Functions

  FunctionDeclaration [Await] : ( Modified )
   FunctionOverloads [?Await]optFunctionImplementation [?Await]

  FunctionOverloads [Await] :
   FunctionOverloads [?Await]optFunctionOverload [?Await]

  FunctionOverload [Await] :
   functionBindingIdentifier [?Await]CallSignature;

  FunctionImplementation [Await] :
   functionBindingIdentifier [?Await]CallSignature{FunctionBody}
   async [no LineTerminator here] functionBindingIdentifier [?Await]CallSignature [Await,AsyncParameter]{FunctionBody [Await]}

A.4 Classes

  MemberFunctionImplementation:
   AccessibilityModifieroptstatic optPropertyNameCallSignature{FunctionBody}
   AccessibilityModifieroptstatic optasync [no LineTerminator here] PropertyNameCallSignature [Await,AsyncParameter]{FunctionBody [Await]}

B Helper Functions

There are two helper functions that are used for Async Functions. The __awaiter helper function is used by both the ES6 as well as the ES5/ES3 transformations. The __generator helper function is used only by the ES5/ES3 transformation.

B.1 __awaiter helper function

var __awaiter = __awaiter || function (g) {  
    function n(r, t) {  
        while (true) {  
            if (r.done) return r.value;  
            if (r.value && typeof (t = r.value.then) === "function")  
                return t.call(r.value, function (v) { return n(g.next(v)) }, function (v) { return n(g["throw"](v)) });  
            r = g.next(r.value);  
        }  
    }  
    return n(g.next());  
};

B.2 __generator helper function

var __generator = __generator || function (m) {  
    var d, i = [], f, g, s = { label: 0 }, y, b;  
    function n(c) {  
        if (f) throw new TypeError("Generator is already executing.");  
        switch (d && c[0]) {  
            case 0 /*next*/: return { value: void 0, done: true };  
            case 1 /*throw*/: throw c[1];  
            case 2 /*return*/: return { value: c[1], done: true };  
        }  
        while (true) {  
            f = false;  
            switch (!(g = s.trys && s.trys.length && s.trys[s.trys.length - 1]) && c[0]) {  
                case 1 /*throw*/: i.length = 0; d = true; throw c[1];  
                case 2 /*return*/: i.length = 0; d = true; return { value: c[1], done: true };  
            }  
            try {  
                if (y) {  
                    f = true;  
                    if (typeof (b = y[c[0]]) === "function") {  
                        b = b.call(y, c[1]);  
                        if (!b.done) return b;  
                        c[0] = 0 /*next*/, c[1] = b.value;  
                    }  
                    y = undefined;  
                    f = false;  
                }  
                switch (c[0]) {  
                    case 0 /*next*/: s.sent = c[1]; break;  
                    case 3 /*yield*/: s.label++; return { value: c[1], done: false };  
                    case 4 /*yield**/: s.label++; y = c[1]; c[0] = 0 /*next*/; c[1] = void 0; continue;  
                    case 6 /*endfinally*/: c = i.pop(); continue;  
                    default:  
                        if (c[0] === 1 /*throw*/ && s.label < g[1]) { s.error = c[1]; s.label = g[1]; break; }  
                        if (c[0] === 5 /*break*/ && (!g || (c[1] >= g[0] && c[1] < g[3]))) { s.label = c[1]; break; }  
                        s.trys.pop();  
                        if (g[2]) { i.push(c); s.label = g[2]; break; }  
                        continue;  
                }  
                f = true;  
                c = m(s);  
            } catch (e) {  
                y = void 0;  
                c[0] = 1 /*throw*/, c[1] = e;  
            }  
        }  
    }  
    return {  
        next: function (v) { return n([0 /*next*/, v]); },  
        1 /*throw*/: function (v) { return n([1 /*throw*/, v]); },  
        2 /*return*/: function (v) { return n([2 /*return*/, v]); },  
    };  
};

B.2.1 _generator Arguments

argument description
m State machine function.

B.2.2 n Arguments

argument description
c The current instruction.

B.2.3 Variables

variable description
d A value indicating whether the generator is done executing.
i A stack of instructions (see below) pending execution.
f A value indicating whether the generator is executing, to prevent reentry (per ES6 spec).
g The region at the top of the s.trys stack.
s State information for the state machine function.
s.label The label for the next instruction.
s.trys An optional stack of protected regions for try..catch..finally blocks.
s.sent A value sent to the generator when calling next.
s.error An error caught by a catch clause.
y The current inner generator to which to delegate generator instructions.
b The method on n to execute for the current instruction. One of "next", "throw", or "return".

B.2.4 Instructions

instruction args description
0 /*next*/ 0-1 Begin or resume processing with an optional sent value.
1 /*throw*/ 1 Throw an exception at the current instruction. Executes any enclosing catch or finally blocks.
2 /*return*/ 0-1 Returns a value from the current instruction. Executes any enclosing finally blocks.
3 /*yield*/ 0-1 Suspends execution and yields an optional value to the caller of the generator.
4 /*yield**/ 1 Delegates generator operations to the provided iterable (not needed for Async Functions, but provided for future support for down-level Generator Functions).
5 /*break*/ 1 Jumps to a labeled instruction. Executes any enclosing finally blocks if the target is outside of the current protected region.
6 /*endfinally*/ 0 Marks the end of a finally block so that the previous break, throw, or return instruction can be processed.

B.2.5 Protected Regions

A protected region marks the beginning and end of a try..catch or try..finally block. Protected regions are pushed onto the s.trys stack whenever a protected region is entered when executing the state machine. A protected region is defined using a quadruple in the following format:

field required description
0 yes The start of a try block.
1 no The start of a catch block.
2 no The start of a finally block.
3 yes The end of a try..catch..finally block.

B.3 __generator helper function (alternate)

var __generator = __generator || function (body) {  
    var done, instructions = [], stepping, region, state = { label: 0 }, delegated;  
    function step(instruction) {  
        if (stepping) throw new TypeError("Generator is already executing.");  
        if (done) {  
            switch (instruction[0]) {  
                case 0 /*next*/: return { value: void 0, done: true };  
                case 1 /*throw*/: throw instruction[1];  
                case 2 /*return*/: return { value: instruction[1], done: true };  
            }  
        }  
        while (true) {  
            stepping = false;  
            var region = state.trys && state.trys.length && state.trys[state.trys.length - 1];  
            if (region) {  
                switch (instruction[0]) {  
                    case 1 /*throw*/:  
                        instructions.length = 0;  
                        done = true;  
                        throw instruction[1];  
                    case 2 /*return*/:  
                        instructions.length = 0;  
                        done = true;  
                        return { value: instruction[1], done: true };  
                }  
            }  
            try {  
                if (delegated) {  
                    stepping = true;  
                    var callback = delegated[instruction[0]];  
                    if (typeof callback === "function") {  
                        var result = callback.call(delegated, instruction[1]);  
                        if (!result.done) return result;  
                        instruction[0] = 0 /*next*/;  
                        instruction[1] = result.value;  
                    }  
                    delegated = undefined;  
                    stepping = false;  
                }  
                switch (instruction[0]) {  
                    case 3 /*yield*/:  
                        state.label++;  
                        return { value: instruction[1], done: false };  
                    case 4 /*yield**/:  
                        state.label++;  
                        delegated = instruction[1];  
                        instruction[0] = 0 /*next*/;  
                        instruction[1] = void 0;  
                        continue;  
                    case 0 /*next*/:  
                        state.sent = instruction[1];  
                        break;  
                    case 6 /*endfinally*/:  
                        instruction = instructions.pop();  
                        continue;  
                    default:  
                        if (instruction[0] === 5 /*break*/ && (!region || (instruction[1] >= region[0] && instruction[1] < region[3]))) {  
                            state.label = instruction[1];  
                            break;  
                        }  
                        if (instruction[0] === 1 /*throw*/ && state.label < region[1]) {  
                            state.error = instruction[1];  
                            state.label = region[1];  
                            break;  
                        }  
                        state.trys.pop();  
                        if (region[2]) {  
                            instructions.push(instruction);  
                            state.label = region[2];  
                            break;  
                        }  
                        continue;  
                }  
                stepping = true;  
                instruction = body(state);  
            } catch (e) {  
                delegated = void 0;  
                instruction[0] = 1 /*throw*/, instruction[1] = e;  
            }  
        }  
    }  
    return {  
        next: function (v) { return step([0 /*next*/, v]); },  
        "throw": function (v) { return step([1 /*throw*/, v]); },  
        "return": function (v) { return step([2 /*return*/, v]); },  
    };  
};
@rbuckton rbuckton added Suggestion An idea for TypeScript Spec Issues related to the TypeScript language specification labels Jan 14, 2015
@rbuckton rbuckton changed the title Spec Proposal: Async Functions Proposal: Async Functions Jan 14, 2015
@MgSam
Copy link

MgSam commented Jan 14, 2015

This proposal looks great. Really looking forward to having this in TypeScript as asynchrony is so important, yet such a pain, in the JS world. I'm glad to see you're planning on supporting ES3/ES5 with async as well.

@rbuckton
Copy link
Member Author

I've added a section on promise implementation compatibility.

@csnover
Copy link
Contributor

csnover commented Jan 15, 2015

Still thinking about the proposal. One super nitpicky request off the top: is it possible to use human-readable variable names in the generator code instead of single letters? An optimiser needs to run over all code anyway for production, and will take care of shortening variables, so this is just obfuscation. When users examine code to understand and learn things, it will be much easier for them to understand what is going on with proper variable names. (This is an issue with the default generated __extends right now too, which is a much simpler function.)

@bgever
Copy link

bgever commented Jan 15, 2015

How are failed or rejected promises handled with this proposal?

I guess you'd need to handle them with try/catch?

@rbuckton
Copy link
Member Author

@bgever, that is correct. In an Async Function you could change from writing this:

function logCommits(): Promise<void> {
  var client = new HttpClient();
  return client.getAsync('https://github.com/Microsoft/TypeScript/commits/master.atom').then(
    response => {
      console.log(response.responseText); 
    }, 
    error => {
      console.log(error); 
    });
}

To this:

async function logCommits(): Promise<void> {
  var client = new HttpClient();
  try {
    var response = await client.getAsync('https://github.com/Microsoft/TypeScript/commits/master.atom');
    console.log(response.responseText);
  }
  catch (error) {
    console.log(error);
  }
}

This also allows for more complex async logic that is harder to achieve with Promise chaining, such as awaiting a promise in a loop:

async function poll(): Promise<void> {
  do {
    var dataReady = await isDataReadyAsync();
  } while (!dataReady);
}

@bgever
Copy link

bgever commented Jan 15, 2015

Thanks for the quick response. Something interesting in your first example is that there's no need to return the result of the promise like before, since the async keyword will trigger this. A common mistake that can be prevented with async/await.

What I dislike about the catch clause, is that you can't use the functional programming way of just passing the function. E.g: .then(successHandler, errorHandler).

Would it be useful to have an inline option as well?

function errorHandler(e) { /* handle */ }
//---or---
var errorHandler = (e) => { /* handle */ };
// Handle errors inline without the need to wrap with try{}.
var result = await myAsync() catch errorHandler;
//---or---
var result = await myAsync() catch e => {
  /* handle */
}

The errorHandler of the inline catch could even return a promise, which would allow the result of await to recover, if that new promise is resolved, or still throw an exception in case of rejection or when no promise is returned from the error handler.

var result = await myAsync() catch async e => { // `async` inline catch returns promise
  if(/*can't recover*/){
    throw new Error('Cannot recover');
  }
  return 'fallback value';
}
//---or---
var result = await myAsync() catch async e => /*can't recover*/ ? throw new Error('Cannot recover') : 'fallback value';

@johnnyreilly
Copy link

Looks good - I agree with the variable naming suggestion by @csnover

@rbuckton
Copy link
Member Author

@bgever, some Promise implementations, such as the native implementation in ES6, have a catch method you could use here:

var result = await myAsync().catch(errorHandler);
//---or---
var result = await myAsync().catch(e => {
  // ...
});

@Steve-Fenton
Copy link

👍

@bgever
Copy link

bgever commented Jan 15, 2015

@rbuckton, ah, of course! As long as we feed a Promise to await - including chained ones - it should work. Thanks for pointing that out.

@fletchsod-developer
Copy link

If I understand correctly, await is part of ES7 specification and async is part of ES6 specification.

I will be happy if TypeScript support await as a make-up wrapper for ES6 because ES6 been long, long overdue and nobody wanna wait for ES7 to come out. We have been patiently waiting and still stuck waiting for most web browser products to start supporting ES7 & ES6.

http://www.joezimjs.com/javascript/synchronizing-asynchronous-javascript-es7/

http://jakearchibald.com/2014/es7-async-functions/

@DanielRosenwasser
Copy link
Member

await is part of ES7 specification and async is part of ES6 specification.

Actually, Promises and the yield keyword are part of the ES6 specification draft. await is a future reserved keyword in the context of a module in the ES6 specification draft.

async/await are part of the ES7 proposal.

@mhegazy
Copy link
Contributor

mhegazy commented Jan 15, 2015

Here is the ES7 proposal:
http://wiki.ecmascript.org/doku.php?id=strawman:async_functions

@masammut
Copy link

What if the instruction string constants ("next", "throw", ...) are replaced by using properties from the following object:

var __instruction = {
  next: 0,
  throw: 1,
  return: 2,
  yield: 3,
  yieldstar: 4,
  break: 5,
  endfinally: 6
}

This would make the generated code more minification and obfuscation friendly, whilst at the same time keeping the unminified code readable.

For example, the following:

// async.js  
var __awaiter = /* ... */;  
var __generator = /* ... */;  

function fn() {  
    var i;  
    return new Promise(function (_resolve) {  
        resolve(__awaiter(__generator(function (_state) {  
            switch (_state.label) {  
                case 0: return ["yield", p0];  
                case 1:  
                    i = _state.sent;  
                    return ["yield", p1];  
                case 2:  
                    return ["return", _state.sent + i];  
            }  
        }));  
    });  
}

Would become:

// async.js  
var __awaiter = /* ... */;  
var __generator = /* ... */;  
var __instruction = /* ... */;

function fn() {  
    var i;  
    return new Promise(function (_resolve) {  
        resolve(__awaiter(__generator(function (_state) {  
            switch (_state.label) {  
                case 0: return [__instruction.yield, p0];  
                case 1:  
                    i = _state.sent;  
                    return [__instruction.yield, p1];  
                case 2:  
                    return [__instruction.return, _state.sent + i];  
            }  
        }));  
    });  
}

@Arnavion
Copy link
Contributor

+1 for compiling ES7 async/await to ES6 yield/Promise/__awaiter

-1 for ES5 and ES3 state-machine emits. I think those are better served with dedicated projects like regenerator instead of bundling into the TS compiler and duplicating effort.

@basarat
Copy link
Contributor

basarat commented Jan 18, 2015

-1 for ES5 and ES3 state-machine emits. I think those are better served with dedicated projects like regenerator instead of bundling into the TS compiler and duplicating effort.

I disagree, I'd rather have TS have this built in instead of forcing me to compile ALL my code to es6 and pushing it to some other transpiler. @Arnavion I expected you would agree : #1641 (comment)

@Arnavion
Copy link
Contributor

@basarat Supporting class inheritance is a tiny rewrite compared to the state-machine rewrite that supporting generators requires. That's why I don't think tsc should take the burden of supporting downlevel emit for generators.

See regenerator's code for yourself and see how complicated the rewrite is.

@basarat
Copy link
Contributor

basarat commented Jan 18, 2015

@Arnavion I've seen you write much more complex algorithms (this is me praising your awesome code skills).

@rbuckton
Copy link
Member Author

@masammut I think a better alternative may be to use integer literals, so where you see "yield" today, we might replace it with 3 /* yield */, which would leave it somewhat readable, and minifiers can strip out the comment.

@Arnavion
Copy link
Contributor

Again, let me emphasize the difference between supporting downlevel emit for this vs something like inheritance. The inheritance support is two parts, the __extends function and the places where it's called. The places where it's called is a simple function call, and the __extends function itself is overridable by client code.

For generators, the two parts are the __generator function and the state machine that it drives. The latter is vastly more complicated than a function call ala class inheritance and is not overridable. If it has a bug, you're stuck with it until it's fixed in the next version of TS. If you encounter a bug in TS's emit for destructuring, you can just not use destructuring. If you encounter a bug in TS's emit for yield, you have to either implement a state machine yourself to minimize change in the generator's callers, or change that function and every caller to a different design (callbacks, etc.)

The generated state machine has to handle conditionals, loops, exceptions, jump labels, ... everything that affects flow control. See the history of the regenerator code I linked, and the referenced bugs and test cases, and think about whether you'd want to report such bugs to TS. The latest fix is less than 2 weeks old, and there are still open issues, so it's by no means a completely solved problem.

I have no doubt the TS team can implement a rewriter that can fix all those known problems and pass all current tests. The question is: Is this something they should spend time on implementing, and then spend time on maintaining? We already have traceur and regenerator (6to5 also uses regenerator, as I just discovered) with their own pluses and minuses (traceur supports all of ES6, regenerator has a better codegen and uses esprima). Do we really need a third that only works with TS?

(It would be awesome that TS supported such a downlevel emit by using regenerator, but this is prevented of course by TS not depending on node to run.)

@johnnyreilly
Copy link

If memory serves the CoffeeScript guys refused to take a PR that covered similar ground. The result was a fork of the project called Iced CoffeeScript: http://maxtaco.github.io/coffee-script/

I'm not sure if es5 / es3 support is worth the trouble because of the debugging story. I value the TS being not too different from the JS. It wouldn't be in this case and so even if available it would probably be a feature I chose not to use with es3 / es5

@ivogabe
Copy link
Contributor

ivogabe commented Jan 18, 2015

For es3 & 5, it might be cleaner to use a callback based emit instead of a state machine? I think an emit like this would be a lot cleaner & more readable:

// TypeScript
var p0: Promise<number> = /* ... */;  
var p1: Promise<number> = /* ... */;  
async function fn() {  
    var i = await p0;  
    return await p1 + i;  
}
// JavaScript
function fn() {
    var i;
    return Promise.resolve(void 0).then(_a);
    function _a() {
        return p0.then(_b);
    }
    function _b(value) {
        i = value;
        p1.then(_c);
    }
    function _c(value) {
        return value + i;
    }
}

// TypeScript
var p0: Promise<number> = /* ... */;  
var p1: Promise<number> = /* ... */;
async function fn() {  
    try {  
        await p0;  
    }  
    catch (e) {
        if (foo) return 42;
        if (bar) return await p1;
        alert(e.message);

        if (baz) {
            await p1;
        }
    }  
    finally {  
        await p1;  
    }  
}

// JavaScript
function __asyncTryCatch(_try, _catch, _finally) {
    Promise.resolve(void 0).then(_try).then(_finally, function(err) {
        return Promise.resolve(_catch(err)).then(function(value) { // value: [boolean, T | Promise<T>]
            if (value[0]) { // return
                return value[1]; // Can be Promise<T> or T.
            } else {
                 // value[1] can be Promise<T> or T, so we need te convert it to Promise<T>
                return Promise.resolve(value[1]).then(_finally);
            }
        }, function(err) {
            Promise.resolve(_finally()).then(function() {
                throw err;
            });
        });
    });
}
function fn() {
    return Promise.resolve(void 0).then(_a);
    function _a() {
        return __asyncTryCatch(_b, _c, _d).then(_e);
    }
    function _b() {
        return p0;
    }
    function _c(e) {
        if (foo) return [true, 42];
        if (bar) return [true, p2];
        alert(e.message);

        if (baz) {
            return [false, p1];
        }
        return [false, void 0];
    }
    function _d() {
        return p1;
    }
    function _e() {
        return void 0;
    }
}

// TypeScript
async function fn() {
    var data = await bar(1);
    if (data.foo) {
        console.log(await bar(2));
        console.log(await bar(3));
    }

    for (var i = 0; i < 10; i++) {
        await bar(4);
    }

    return data;
}
// JavaScript
function fn() {
    var data, i, dataObj;
    return Promise.resolve(void 0).then(_a); // Start in a Promise context, since a Promise catches errors
    function _a() {
        return bar(1).then(_b);
    }
    function _b(value) {
        data = value;
        if (data.foo) {
            return bar(2).then(_c);
        }
        return _e();
    }
    function _c(value) {
        console.log(value);
        return bar(3).then(_d);
    }
    function _d(value) {
        console.log(value);
        return _e();
    }
    function _e() { // Code after the 'if' is wrapped in a function, otherwise it would be emitted twice.
        i = 0;
        return _f();
    }
    function _f() {
        if (i < 10) {
            return bar(4).then(_g);
        }
        return data;
    }
    function _g() {
        i++;
        return _f();
    }
}

A function will be cut in smaller functions. This way there are no nested callback needed.

Advantages:

  • Promises already handle errors (you can throw errors in a then callback).
  • You can easily control the order of execution (with for, if et cetera) with the then methods.
  • No need for big helper functions (only __asyncTryCatch)

@Arnavion
Copy link
Contributor

@ivogabe

The original proposal is for emitting generators in general, not just async-await, so it can't rely on Promises. Generators are synchronous whereas a Promise never resolves synchronously.

Also, your alternative emit is equivalent to the proposed state machine:

  • The proposed state machine has a case-block for each states, while yours has a closure for each state.
  • A state in the proposed state machine would return a label for the next state, while yours does a .then() with the next function as the parameter.
  • The proposed state machine is driven by a driver that interprets the state transitions, while yours relies on Promise.then() for that.

If it's a matter of readability, you could apply the same transformation for the generator state machine. Just move each case-block to its own closure, and make each closure return a reference to the next closure to call instead of the label of the next state (i.e., instead of return ["break", 6]; do return _g;). The switch-case based state machine is just the canonical implementation (it's also smaller than wrapping each state into its own closure and, more importantly, doesn't blow the stack for long loops since it's effectively TCO'd). There is no difference in complexity between the two emits (neither for the compiler that needs to emit it, nor for the human that needs to read it) - they both require the same rewrite.

@MatAtBread
Copy link

I totally misunderstood the question, which is about support for async/await in the typescript compiler. I have no insight into that at all

@gaurav21r
Copy link

@MatAtBread @jamiewinder As @mhegazy mentioned, TypeScript 1.7+ has support for async/await but for es6 targets (ie. the output will use generators). In the roadmap for 2.0, the proposal is that TypeScript whould support async/await for ES3/ES5. Though personally, with the rate of JS environments compliying with ES6 https://kangax.github.io/compat-table/es6/ , I don't think it will be relevant anymore except for legacy environments.

@RichiCoder1
Copy link

Do not underestimate the sway of legacy environments.

@gaurav21r
Copy link

@RichiCoder1 I know! I've been there, but I guess as a community we have always felt IE drags us down. That's changing with Microsoft officially deprecating older IE. The support for generators is pretty good at this point. Only IE 11 and Safari lag behind.

I know this is still a significant audience, especially with Enterprise but looking over the above discussion, the Cost - benefit analysis of async/await emitting ES3/ES5 compatible code in TS will be interesting.

But hey that's just my personal opinion. I'd be really happy if this happens! :)

@mhegazy
Copy link
Contributor

mhegazy commented Feb 22, 2016

Sorry for closing the issue and not leaving a comment.

async/awiat support in the type system and for ES6 target was done in 1.7, and enabled by default with no flags.
async/await support for ES3/ES5 emit is the remaining piece, and that should be better tracked by #1564. this is planned for 2.0.

@RReverser
Copy link
Contributor

@mhegazy Is there a way to disable async/await transformation only? (if I'm targetting Node.js chakra only and don't want this extra step with wrappers)

@mhegazy
Copy link
Contributor

mhegazy commented Mar 3, 2016

No, issue #4692 tracks more fine grained control over transformations.

@jameskeane
Copy link
Contributor

@saschanaz @jods4 For the "typing issue", you can always explicitly declare arity:

module Promise {
  function all<T1>(promises: [T1|Thenable<T1>]): Promise<[T1]>;
  function all<T1, T2>(promises: [T1|Thenable<T1>, T2|Thenable<T2>]): Promise<[T1, T2]>;
  function all<T1, T2, T3>(promises: [T1|Thenable<T1>, T2|Thenable<T2>,
      T3|Thenable<T3>]): Promise<[T1, T2, T3]>;
  function all<T1, T2, T3, T4>(promises: [T1|Thenable<T1>, T2|Thenable<T2>,
      T3|Thenable<T3>, T4|Thenable<T4>]): Promise<[T1, T2, T3, T4]>;
  function all<R>(promises: (R | Thenable<R>)[]): Promise<R[]>;

  // ... to whatever is reasonable
}

@mhegazy
Copy link
Contributor

mhegazy commented Sep 7, 2016

Async function support for target ES3/ES5 is now available in [email protected] and later.

@wiktor-k
Copy link

wiktor-k commented Sep 9, 2016

For people interested in how does the output look like currently I've put a gist here: https://gist.github.com/wiktor-k/eed3ef8032f73caba33d4fd1ddd16308

Is there a plan to use async/await -> ES5 transformation similar to kneden? The output looks a lot simpler there. The TS approach looks like it would be easier to extend to support generators too.

@jods4
Copy link

jods4 commented Sep 9, 2016

@wiktor-k I don't quite agree kneden looks simpler. For the trivial return x.then(y), sure, but in general...

Note that in your gist, _generator and _await are helper functions, emitted once for the whole code base. The code that is actually generated for your async functions are x() and y(), which are not that bad once you understand how they work -- basically, one case for each branch of code in your original function. And because of this, the generated code maps pretty closely to the original code. This nice property is very not true for complex control structures in kneden.

Have you looked at how kneden translates a try/catch/finally with awaits? Brace yourself:

function test() {
  return Promise.resolve().then(function () {
    return Promise.resolve().then(function () {
      return db.destroy();
    }).catch(function (err) {
      return Promise.resolve().then(function () {
        console.log(err);
        return db.post({});
      }).then(function (_resp) {
        console.log(_resp);
      });
    }).then(function () {
      return db.info();
    }, function (_err) {
      return Promise.resolve().then(function () {
        return db.info();
      }).then(function () {
        throw _err;
      });
    });
  }).then(function () {});
}

Personally, I much prefer the y() code generated by TS. I wouldn't be surprised if it allocates less (hence performs better) as well.

As soon as there is complex control flow (loops, try, etc) using Promise only to write async code becomes very tricky. There is no "similar to how a human would do so" as most of my colleagues would be unable to write the code for even a simple loop.

Another important aspect -- but I might be wrong here -- is that it's hard to guarantee kneden supports all valid JS code and produces correct code (homepage says "it tries"). Notably, at this point it doesn't support return from inside switch and try/catch/finally blocks.

@MatAtBread
Copy link

I don't know kneden, but nodent https://github.com/MatAtBread/nodent does implement all JS constructs containing async/await using Promises (or generators, or callbacks) through syntax transformation, and is reasonably mature. It also passes through all ES6 features. The main advantage is performance - it's significantly faster than Promise/generator implementations, including the built-in async/await support in Chrome 54. There's also covers for Babel, browserify and webpack.

Disclaimer: I'm the author of nodent.

@jods4
Copy link

jods4 commented Sep 9, 2016

@MatAtBread I had a look at your project and I was very impressed! 👏
The approach when generating Promises is indeed different from kneden.
I have to admit that I was surprised the generated code performs so well.

I noticed the following in the playground, though. It seems to me that when using Promises the following ES7 code generates ES5 code that results in recursive calls of arbitrary depth (in this case, the ES5 looks like it will nest 3'000'000 function calls, i.e. 3 per loop iteration).

async function x() {
    for (let i = 0; i < 1000000; i++) {
        if (i == 999999) await a;
    }
}

Turning loop iterations into recursive calls has serious drawbacks. If I am right, I think this makes it not ok for a widely used compiler like TS.

@MatAtBread
Copy link

You're quite right about loops without tall-recursion. There's a discussion of the issue in MatAtBread/fast-async#11

The solution (the one I use in almost ask my production code) is to use Promises over pure-ES5 mode.

@MatAtBread
Copy link

... actually, I've only just understood your case - it only awaits rarely. That's a good one, I might try and handle that one differently to avoid the recursion, since that code path doesn't need it. Thanks for the input!

@matAtWork
Copy link

@jods4 - Inspired as I was by your comment above, I (finally) implemented loops using a sync/async trampoline. Performance isn't too bad, since I at least avoid creating Promises and turning ticks if the loop doesn't yield. A long loop (see link below) is around 1.2s on my mac in ES5-eager, 2.8s with native Promises, and 3.1s with generators/Promises. Those numbers are only a rough guide as a lot of factors are not easy to control for in the browser. More reliable results are available from the command line.

I've not released nodent v3 yet as I'm still testing, but I updated the playground for testing.

http://nodent.mailed.me.uk/#%2F*%20Aync%20addition%20*%2F%0Aasync%20function%20add(a%2C%20b)%20%7B%0A%20%20%20%20return%20a%20%2B%20b%3B%0A%7D%0A%0A%2F*%20Run%20the%20async%20addition%20lots%20of%20times%20*%2F%0Aasync%20function%20test()%20%7B%0A%20%20%20%20var%20x%20%3D%200%3B%0A%20%20%20%20for%20(var%20n%20%3D%200%3B%20n%20%3C%205e5%3B%20n%2B%2B)%20%7B%0A%20%20%20%20%20%20%20%20x%20%3D%20await%20add(x%2C%201)%3B%0A%20%20%20%20%7D%0A%20%20%20%20return%20n%20-%20x%3B%0A%7D%0A%0Aasync%20function%20run()%20%7B%0A%20%20%20%20var%20t%20%3D%20Date.now()%3B%0A%20%20%20%20await%20test()%3B%0A%20%20%20%20return%20Date.now()%20-%20t%3B%0A%7D%0A%0A%2F*%20Run%20the%20test%2C%20in%20a%20Generator%20friendly%20way%20from%20non-async%20code.%20*%2F%0Arun().then(log)%3B%0A

@jods4
Copy link

jods4 commented Sep 26, 2016

@matAtWork nice work. I was curious how you'd solve it!
I am still intrigued that this is 30% faster than native async/await despite all the calls and captures!

@matAtWork
Copy link

Probably not the place to spend ages on it, but JS is the proof that "premature optimization is the root of all evil". In particular, a lot of performance hits on older JS engines have been heavily fixed - closures are a perfect example (now almost as fast as object dereferencing), and Promises are, well, promising.

When I started Nodent almost 3 years ago, avoiding Promises was a very easy win (Nodent only used callbacks). Since then, Promise implementations have been honed down, and the JS engines are much better at identifying and optimizing pinch points, so the benefits of avoiding them have fallen.

I'm guessing the same will be true of generators, although it's not there yet - right now they're still 3x-4x slower.

Meanwhile, the kind of transforms Nodent does really are pretty much what everyone in Node-land does by hand (identifying common code to put in functions and pass as callbacks), of which there is so much around I'm sure the engine writers have squeezed lots of performance out of it.

@JohnGalt1717
Copy link

Now that this is added how does one await an array of promises? I have a ton of ng.IPromise and I want to execute them all at the same time and then await them all completing.

@bman654
Copy link

bman654 commented Dec 8, 2016 via email

@jamiewinder
Copy link

jamiewinder commented Dec 8, 2016

Also handy:

const [a, b, c] = await Promise.all([Promise.resolve(1), Promise.resolve(true), Promise.resolve('a')]);
// a is number, b is boolean, c is string

@JohnGalt1717
Copy link

Promise.all doesn't seem to work with ng.IPromise so I'm using $q.all right now which sadly isn't strongly typed on the results. I was hoping that there was a better way.

@RReverser
Copy link
Contributor

@JohnGalt1717 That's weird, it should work judging from ng.IPromise definition. What error do you get / what do you observe?

@microsoft microsoft locked and limited conversation to collaborators Jun 18, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Committed The team has roadmapped this issue ES Next New featurers for ECMAScript (a.k.a. ESNext) Fixed A PR has been merged for this issue Suggestion An idea for TypeScript
Projects
None yet
Development

No branches or pull requests