Adds 'awaited' type to better handle promise resolution and await

[rbuckton]

Following the discussion in microsoft/vscode#30216, I noted that the Promise definition in VSCode was out of date. I created microsoft/vscode#30268 to address this and ran into a number of cases where I had to add explicit type annotations to work around limitations in the checker.

As a result, I discovered two classes of issues with how we handle resolution for Promise-like types:

• Incorrect return types or errors with complex Promise chains created via then().
• Incorrect eager resolution of the "awaited type" for a type variable.

To that end, I am introducing two features into the compiler:

• An awaited T type operator, used to explicitly indicate the "awaited type" of a type
  following the same type resolution behavior as the await expression and providing an explicit
  parallel in the type system for the runtime behavior of await.
• An AwaitedType type in the checker, used to defer resolution of the "awaited type" of a type
  variable until it is instantiated as well as to better support type relationships.

Incorrect return types or errors with complex Promise chains

The then() method on Promise currently has the following definition:

interface Promise<T> {
    then<TResult1 = T, TResult2 = never>(
        onfulfilled?: ((value: T) => TResult1 | PromiseLike<TResult1>) | null, 
        onrejected?: ((reason: any) => TResult2 | PromiseLike<TResult2>) | null
    ): Promise<TResult1 | TResult2>;
}

Generally, this works well if the return statements in the function passed to onfulfilled are simple:

declare const pr: Promise<number>;

pr.then(x => {
    if (x < 0) return "negative";
    return x;
}).then(y => {
    y; // string | number
});

However, the following results in errors:

declare function f(x: number): Promise<string>;
declare const pr: Promise<number>;

// error: Argument of type '(x: number) => number | Promise<string>' is not assignable to parameter
// of type '(value: number) => string | PromiseLike<string>';
pr.then(x => { 
    if (x < 0) return f(x);
    return x;
});

To resolve this, this PR adds a new awaited T type operator that allows a developer to more
reliably indicate the type that would be the result of the implicit unwrapping behavior inherent to both Promise resolution as well as the await expression.

This allows us to update our definition of Promise to the following:

interface Promise<T> {
    then<TResult1 = T, TResult2 = never>(
        onfulfilled?: ((value: awaited T) => TResult1) | null, 
        onrejected?: ((reason: any) => TResult2) | null
    ): Promise<awaited TResult1 | awaited TResult2>;
}

With awaited T, regardless what type you specify for T, the type of value in the
onfulfilled callback will be recursively unwrapped following the same mechanics of await (which
in turn follows the runtime behavior of the ECMAScript specification).

In addition, the much simpler return types of the onfulfilled and onrejected callbacks allow
for more complex logic inside of the bodies of the callbacks. The complex work of unwrapping the
return types of the callbacks is then pushed into the return type of then, where the awaited
types of TResult and TResult2 are resolved.

As such, we can now infer the correct types for the above example:

declare function f(x: number): Promise<string>;
declare const pr: Promise<number>;

pr.then(x => { 
    if (x < 0) return f(x);
    return x;
}).then(y => {
    y; // string | number
});

Incorrect eager resolution of the "awaited type" for a type variable

This brings us to the second issue this PR addresses. The underlying implementation of await in
the type checker already recursively resolves the "awaited type" of an expression. For example:

async function f(x: Promise<Promise<number>>) {
    const y = await x;
    return y; // number
}

declare const a: Promise<Promise<number>>;

f(a).then(value => {
    value; // number
});

However, it can fall short in cases where generics are concerned:

async function f<T>(x: Promise<T>) {
    const y = await x; 
    return y; // T
}

declare const a: Promise<Promise<number>>;

f(a).then(value => { 
   value; // Promise<number> !!!
});

The reason we get the wrong type in the second example is that we eagerly get the "awaited type"
of x, which is T. We cannot further resolve T as it has not been instantiated, so the
return type of f becomes Promise<T>. When we supply a nested promise type to f, we assign
the type MyPromise<number> to T which then instantiates the return type of f(a) as
Promise<MyPromise<number>>!

To address this, TypeScript will now internally have the concept of an AwaitedType which indicates
the deferred resolution of the "awaited type" of a type variable. As a result, whenever you await
an expression whose type is a type variable, or whenever we encounter an awaited T operator for a
type variable, we will internally allocate an AwaitedType for that type variable. Once the type
variable is instantiated, the "awaited type" will be resolved.

With this change, the failing example above now has the correct type:

async function f<T>(x: Promise<T>) {
    const y = await x; 
    return y; // awaited T
}

declare const a: Promise<Promise<number>>;

f(a).then(value => { 
   value; // number
});

This also allows us to more accurately check type relationships between the "awaited types" of
two generics, following these rules:

When relating two types S and T, where S is the source type and T is the target:

• awaited S is related to awaited T if S is related to T. [proven]
• S is related to awaited T if S is related to awaited C, where C is the constraint of T. [disproven]
• awaited S is related to T if awaited C is related to T, where C is the constraint of S. [proven]

Out of scope

One thing this PR does not cover is attempting to recursively unwrap the T of a Promise<T> when
explicitly written as such by the user. Generally this is unnecessary as the T will be
unwrapped when using then() or catch():

const p1 = new Promise<Promise<number>>(resolve => resolve(Promise.resolve(1))); 
const p2 = p1.then(); // Promise<number>

Bear in mind that this means that though p1 in the example above has the type
Promise<Promise<number>>, p2 will have the type Promise<number>. It may seem counter-
intuitive, but this is the correct behavior and mitigates the need to add syntax to unwrap
type parameters.

Definitions

• onfulfilled callback - The first callback passed to the then() method of a promise.
  The callback accepts a single value parameter which is the fulfilled value of the promise.
• fulfillment type - The explicit type of the value parameter for the onfulfilled callback of a promise.
• awaited type - If a type T is promise-like, the awaited type of the fulfillment type of T; otherwise, T.
• recursive unwrapping - The process of resolving the awaited type above.

[weswigham]

Will need to remember to make an issue to remove the --exclude 'src/lib/es2015.iterable.d.ts' --exclude 'src/lib/es2015.promise.d.ts'" from the build once tslint is updated with a nightly that understands the new syntax.

Bikeshed: Is there a reason it's promised T rather than await T (to mirror the expression syntax)? It seems apt, especially when it's used to describe the behavior of the operator - similar to how we reuse this in type positions (even though it's not exactly the same, the concept is analogous).

[KiaraGrouwstra]

Problem 1 may be among the many issues that could also be solved given #6606:

interface UnwrapPromise {
  <U>(v: PromiseLike<U>): UnwrapPromise(U); // try unwrapping further, just in case
  <U>(v: U): U;
  // ^ else case: can't unwrap, so no-op
};
// ^ function type used to unwrap promise types
interface Promise<T> {
    then<TResult1 = T, TResult2 = never>(
        onfulfilled?: ((value: T) => TResult1) | undefined | null, 
        onrejected?: ((reason: any) => TResult2) | undefined | null
    ): Promise<UnwrapPromise(TResult1) | UnwrapPromise(TResult2)>;
}

Since this approach does not rely on TResult1 | PromiseLike<TResult1> for unwrapping, a return type like number | Promise<string> should automatically pass through UnwrapPromise yielding number | string, as intended.

I'm not so much expecting the approach to change for this current in-progress PR; I'm just under the impression that particular proposal has been somewhat under-appreciated given what it could address.

Edit: this UnwrapPromise might address 2 as well. The great advantage of putting logic using typings is that you can be sure it only gets evaluated when any used generics have been resolved as well, potentially making solutions less complex than they might be if tackled on the compiler level.

[masaeedu]

It'd be really nice if you could solve these problems by making the overall type system sufficiently expressive, rather than adding magic user-inaccessible types.

[gcnew]

I concur with @masaeedu. It's feels like a brute force/patchy solution that might become a considerable debt in the near future.

[rbuckton]

@masaeedu, my intention here is to make the type system more expressive. This introduces a type-system parallel to the await expression, similar to keyof in the type system being a parallel to in and Object.keys at runtime. Addressing the second issue without the first would have truly introduced a user-inaccessible type.

With this change, the type system would have a mechanism to reflect at design time the behavior of both the await expression and Promise resolution that is both internally consistent and able to be expressed by the user. The previous approach of using T | PromiseLike<T> was more akin to using type-system magic to extract the correct T, but that approach only works for inference. I believe having an explicit and well-defined mechanism for reflecting this behavior at design time is the correct direction, and allows other library authors to leverage the same type language for non-native Promise implementations.

[masaeedu]

@rbuckton There is nothing in particular that is special about Promise, which is why it seems weird to make a promise T keyword instead of fixing the problems in the type system that make it impossible to express unwrapping of monadic types like Promise. You'll have people complaining about the same problem with things like nested lists or sets or whatever.

[rbuckton]

@masaeedu That's not what this proposes. It is not promise T, but rather promised T, which is intended to mean "get the underlying value of T after all recursive unwrapping of a Promise is applied".

JavaScript promises do not let you have a Promise for a Promise. Any time a Promise is settled with another Promise, the first Promise adopts the state of the second promise. This effectively unwraps the second Promise's value. Strongly typed languages like C# do not have this concept. Consider Task<T>: If you create a Task<Task<string>>, the result is Task<string> as the T cannot be unwrapped. Without a syntactic construct to represent this behavior, any other attempt to resolve this in the type system is "magic".

By introducing promised T, regardless of whether T is number, or Promise<number>, or Promise<Promise<number>>, the result of promised T would be number.

Perhaps @weswigham is correct and we need to bikeshed the keyword. Maybe promisedof T, or awaited T, etc. would be more explanatory.

Also, while Promise may be monadic, a promise-like is not. The unwrap/adopt behavior is not guaranteed in a promise-like, but that doesn't matter to a Promise, as it will continually adopt the state of any promise-like provided as the fulfillment value of the onfulfilled callback.

[gcnew]

@masaeedu The real problem is that then does an implicit join based on the runtime value. While it makes sense for a dynamically typed language such as JS, it's really bad from a type system perspective (a separate map and bind would have been much better). Now, having these wonky runtime semantics, a patchy solution for Promise looks good at first, but I think it will become redundant and may even hinder future development if features such as conditional logic (e.g. #12424, #17077 (comment)) get implemented.

[masaeedu]

@rbuckton Sorry, promise is a typo on my part, but I do understand what this is proposing: "express unwrapping of monadic types like Promise", i.e. getting T from Promise<T>. My point is that the unwrapping you are referring to, and the fact that you need to prevent Promise<Promise<T>> is not a special feature of promises: the Promise.then method is an instance of the more general monadic bind operation, which "flat maps" over the monadic type.

If you allow for type-level functions as @tycho01 has shown, you can express recursive unwrapping of the Promise type and a great number of other features besides. The fact that it becomes possible for users to correctly express flatmap over promises is simply emergent from that feature. In addition, unlike this promised keyword, that concept is agnostic to Promises themselves, it works for any monadic type; whether I'm trying to express a "list-that-is-always-flat" type, or a set, or any number of such things.

[masaeedu]

@gcnew With type-level functions, we already have a powerful way to encode conditional logic or base-cases for pattern matching on types: overloaded functions. This is actually very close to how type families in Haskell work, so I feel it is good enough, but I guess whether or not we need some other mechanism of conditional matching on types is a debate for a different issue (probably #12424).

[KiaraGrouwstra]

whether or not we need some other mechanism of conditional matching on types is a debate for a different issue (probably #12424)

My hunch is also that with { 0: a, 1: b }[c] plus the demonstrated overloaded functions we might already have enough to cover conditional logic... which isn't to say no additional syntax sugar like c ? a : b could be added, but I guess with 6606 we could just make type If<Cond, Then, Else> = { (v: true) => Then; (v: false) => Else }(Cond); anyway.

Edit: to ensure lazy evaluation you can't actually abstract that pattern into that If as evaluating an inappropriate branch could yield you an error, so that could make a case for a ? b : c syntax sugar there, though in all likelihood so many cases could settle for strict evaluation the gains in sweetening the remaining cases a bit further seem marginal.

[rbuckton]

Open items from design meeting on 2017-08-04:

• Review high-order type relationships for correctness.
• Bikeshed on operator name. Options discussed included:
  • promised T (current)
  • promisedof T
  • await T
  • fulfilled T

[rbuckton]

In the design meeting on 2017-08-25 we decided on the name awaited T

[KiaraGrouwstra]

I now got to the point of being able to try that UnwrapPromise idea for a more general solution. Unfortunately it is not automatically union-proof as I had hoped. Trying to think of potential solutions.

Edit: guess we knew the solution, i.e. separately calculating return types for different union constituents, see #17471 (comment). I suppose these type calls would have use for that -- I should try for a bit.

Edit 2: looks like iterating over the options as suggested there did it, addressing the issue raised here without special-casing Promise. It's only iterating over them in type calls, so performance for existing code is unaffected.

[KiaraGrouwstra]

if #21613 lands, that should probably address the issues raised here.

[rbuckton]

@tycho01, we discussed this in our last design meeting and unfortunately there are still issues with higher-order assignability between generic Promise<Awaited<T>> and Promise<T> that aren't currently resolvable without some kind of additional syntax. It may be that a syntactic awaited T is still necessary.

[mhegazy]

With conditional types in now, we might want to rethink our approach here. closing the PR for house keeping purposes since it is unactionable at the moment, but leaving the branch for further investigations.

[rbuckton]

We may still need to pursue this approach given the fact that conditional types cannot solve this.