Placeholder Type Declarations

[DanielRosenwasser]

Background

There are times when users need to express that a type might exist depending on the environment in which code will eventually be run. Typically, the intent is that if such a type can be manufactured, a library can support operations on that type.

One common example of this might be the Buffer type built into Node.js. A library that operates in both browser and Node.js contexts can state that it handles a Buffer if given one, but the capabilities of Buffer aren't important to the declarations.

export declare function printStuff(str: string): void;

/**
 * NOTE: Only works in Node.js
 */
export declare function printStuff(buff: Buffer): void;

One technique to get around this is to "forward declare" Buffer with an empty interface in the global scope which can later be merged.

declare global {
    interface Buffer {}
}

export declare function printStuff(str: string): void;

/**
 * NOTE: Only works in Node.js
 */
export declare function printStuff(buff: Buffer): void;

For consuming implementations, a user might need to say that a type not only exists, but also supports some operations. To do so, it can add those members appropriately, and as long as they are identical, they will merge correctly. For example, imagine a library that can specially operate on HTML DOM nodes.

function printStuff(node: HTMLElement) {
    console.log(node.innerText);
}

A user might be running in Node.js or might be running with "lib": ["dom"], so our implementation can forward-declare HTMLElement, while also declaring that it contains innerText.

declare global {
    interface HTMLElement {
        innerText: string;
    }
}

export function printStuff(node: HTMLElement) {
    console.log(node.innerText);
}

Issues

Using interface merging works okay, but it has some problems.

Conflicts

Interface merging doesn't always correctly resolve conflicts between declarations in two interfaces. For example, imagine two declarations of Buffer that merge, where a function that takes a Buffer expects it to have a toString property.

If both versions of toString are declared as a method, the two appear as overloads which is slightly undesirable.

declare global {
    interface Buffer {
        // We only need 'toString'
        toString(): string;
    }
}

export function printStuff(buff: Buffer) {
    console.log(buff.toString());
}

////
// in @types/node/index.d.ts
////

interface Buffer {
    toString(encoding?: string, start?: number, end?: number): string;
}

Alternatively, if any declaration of toString is a simple property declaration, then all other declarations will be considered collisions which will cause errors.

declare global {
    interface Buffer {
        toString(): string
    }

}

////
// in @types/node/index.d.ts
////

interface Buffer {
    toString: (encoding?: string, start?: number, end?: number) => string;
}

The former is somewhat undesirable, and the latter is unacceptable.

Limited to Object Types

Another problem with the trick of using interfaces for forward declarations is that it only works for classes and interfaces. It doesn't work for, say, type aliases of union types. It's important to consider this because it means that the forward-declaration-with-an-interface trick breaks as soon as you need to convert an interface to a union type. For example, we've been taking steps recently to convert IteratorResult to a union type.

Structural Compatibility

An empty interface declaration like

interface Buffer {}

allows assignment from every type except for unknown, null, and undefined, because any other type is assignable to the empty object type ({}).

Proposal

Proposed is a new construct intended to declare the existence of a type.

exists type Foo;

A placeholder type declaration acts as a placeholder until a type implementation is available. It provides a type name in the current scope, even when the concrete implementation is unknown. When a non-placeholder declaration is available, all references to that type are resolved to an implementation type.

The example given is relatively simple, but placeholder types can also support constraints and type parameters.

// constraints
exists type Foo extends { hello: string };

// type parameters
exists type Foo<T>;

// both!
exists type Foo<T, U> extends { toString(): string };

A formal grammar might appear as follows.

PlaceholderTypeDeclaration ::

  exists _{[No LineTerminator here]} type BindingIdentifier TypeParameters_opt Constraint_opt ;

Implementation Types

A placeholder type can co-exist with what we might call an implementation type - a type declared using an interface, class, or type alias with the same name as the placeholder type.

In the presence of an implementation type, a placeholder defers to that implementation. In other words, for all uses of a type name that references both a placeholder and an implementation, TypeScript will pretend the placeholder doesn't exist.

Upper Bound Constraints

A placeholder type is allowed to declare an upper bound, and uses the same syntax as any other type parameter constraint.

exists type Bar extends { hello: string };

This allows implementations to specify the bare-minimum of functionality on a type.

exists type Greeting extends {
    hello: string;
}

function greet(msg: Greeting) {
    console.log(msg.hello);
}

If a constraint isn't specified, then the upper bound is implicitly unknown.

When an implementation type is present, the implementation is checked against its constraint to see whether it is compatible. If not, an implementation should issue an error.

exists type Foo extends {
    hello: string
};

// works!
type Foo = {
    hello: string;
    world: number;
};

exists type Bar extends {
    hello: string;
}

// error!
type Bar = {
    hello: number; // <- wrong implementation of 'hello'
    world: number;
}

Type Parameters

A placeholder type can specify type parameters. These type parameters specify a minimum type argument count for consumers, and a minimum type parameter count for implementation types - and the two may be different!

For example, it is perfectly valid to specify only type arguments which don't have defaults at use-sites of a placeholder type.

exists type Bar<T, U = number>;

// Acceptable to omit an argument for 'U'.
function foo(x: Bar<string>) {
    // ...
}

But an implementation type must declare all type parameters, even default-initialized ones.

exists type Bar<T, U = number>;

// Error!
// The implementation of 'Bar' needs to define a type parameter for 'U',
// and it must also have a default type argument of 'number'.
interface Bar<T> {
    // ...
}

Whenever multiple placeholder type or implementation type declarations exist, their type parameter names must be the same.

Different instantiations of placeholders that have type parameters are only related when their type arguments are identical - so for the purposes of variance probing, type parameters are considered invariant unless an implementation is available.

Relating Placeholder Types

Because placeholder types are just type variables that recall their type arguments, relating placeholders appears to fall out from the existing relationship rules.

The intent is

• Two instantiations of the same placeholder type declaration are only related when their type arguments are identical.
• A placeholder type is assignable to any type whose constraint is a subtype of the target.

In effect, two rules in any of our type relationships should cover this:

• S and T are identical types.
• S is a type parameter and the constraint of S is [[related to]] T.

Merging Declarations

Because different parts of an application may need to individually declare that a type exists, multiple placeholder types of the same name can be declared, and much like interface declarations, they can "merge" in their declarations.

exists type Beetlejuice;
exists type Beetlejuice;
exists type Beetlejuice;

In the event that multiple placeholder types merge, every corresponding type parameter must be identical. On the other hand, placeholder constraints can all differ.

interface Man { man: any }
interface Bear { bear: any }
interface Pig { pig: any }

exists type ManBearPig extends Man;
exists type ManBearPig extends Bear;
exists type ManBearPig extends Pig;

When multiple placeholder types are declared, their constraints are implicitly intersected to a single upper-bound constraint. In our last example, ManBearPig's upper bound is effectively Man & Bear & Pig. In our first example with Beetlejuice, the upper bound is unknown & unknown & unknown which is just unknown.

Prior Art

C and C++ also support forward declarations of types, and is typically used for opaque type handles. The core idea is that you can declare that a type exists, but can never directy hold a value of that type because its shape/size is never known. Instead, you can only deal with pointers to these forward declared types.

struct FileDescriptor;

FileDescriptor* my_open(char* path);
void my_close(FileDescriptor* fd);

This allows APIs to abstract away the shape of forward-declared types entirely, meaning that the size/shape can change. Because these can only be pointers, there isn't much you can do with them at all (unlike this implementation).

Several other programming languages also support some concept of "opaque" or "existential" types, but are generally not used for the same purposes. Java has wildcards in generics, which is typically used to allow one to say only a bit about how a collection can be used (i.e. you can only write Foos to some collection, or read Bars, or you can do absolutely nothing with the elements themselves). Swift allows return types to be opaque in the return type by specifying that it is returning some SuperType (meaning some type variable that extends SuperType).

FAQ and Rationale

Why can placeholder types have multiple declarations with different constraints?

We have two "obvious" options.

1. Enforce that constraints are all identical to each other.
2. Allow upper-bound constraints to be additive. This is effectively like intersecting the constraints so that a given type implementation has to satisfy all placeholder declarations.

I believe that additive constraints are the more desirable behavior for a user. The idea is that different parts of your application may need different capabilities, and given that interfaces can already model this with interface merging, using intersections provides a similar mechanism.

Are these just named bounded existential types?

In part, yes! When no implementation type exists, a placeholder type acts as a bounded existential type variable.

Sorry I'm not sure what you're talking about. Please move along and don't write blog posts about how TypeScript is adding bounded existential types.

Can placeholder types escape scopes?

function foo() {
    exists type Foo;
    return null as any as Foo;
}

Maybe! It might be possible to disallow placeholder types from escaping their declaring scope. It might also be reasonable to say that a placeholder can only be declared in the top level of a module or the global scope.

Do we need the exists keyword?

Maybe we don't need the exists keyword - I am open to doing so, but wary that we are unnecessarily abusing the same syntax. I'd prefer to be explicit that this is a new concept with separate syntax, but if we did drop the exists, we would change the grammar to the following.

PlaceholderTypeDeclaration ::

  type _{[No LineTerminator here]} BindingIdentifier TypeParameters_opt Constraint_opt ;

[dragomirtitian]

What happens if no implementation type is found? Can we assign anything to a placeholder type reference?

For example:

exists type Buffer extends { toArray(): number[] }
export function printStuff(buff: Buffer) {
    console.log(buff.toString());
}
printStufff({ toArray() { return [0] } }); // error here ?

[DanielRosenwasser]

What happens if no implementation type is found? Can we assign anything to a placeholder type reference?

In your example only Buffer is assignable to Buffer (unless you have another placeholder type which extends Buffer. It pretty much acts like a type parameter.

So to answer your question

• if no implementation type found, the type acts like a type parameter
• only other placeholder types and type parameters which are constrained to some placeholder type P is assignable to P.

[rbuckton]

I'm not a fan of exists as a keyword, but I suppose it's no worse than declare. It does seem like you are missing a form for declaring a module exists as well as types in that module, so as not to introduce an ambient module incorrectly:

exists module "net" {
  exists type Server;
} 

export function connect(srv: import("net").Server) {...}

[rbuckton]

Alternatively:

exists type Server from "net";

[DanielRosenwasser]

@rbuckton I think you can already do that with module augmentations and ambient module declarations which merge.

// globals.d.ts
declare module "net" {
    export exists type Server;
}

// consumer.ts
import net = require("net");

function connect(srv: net.Server): {
    // ...
}

[rbuckton]

Two problems with that:

• Module augmentations in a .d.ts don't "augment" but define.
• Module augmentations in a .ts error if the module isn't already defined.

Just as the goal with exists type X is not to define but declare a placeholder, I don't want to define "net" in my example but declare a placeholder.

I don't want to introduce a "net" module in my declaration file that makes import ... from "net" work in your project even when you don't have the NodeJS types installed.

[fatcerberus]

Sorry I'm not sure what you're talking about. Please move along and don't write blog posts about how TypeScript is adding bounded existential types.

Aren’t they literally existential types though? You’re introducing an opaque type variable in an otherwise non-generic context, to stand in for any type it needs to be; (my understanding is) existential types do the same, allowing parameterization on a type without T contributing to the overall type of the function/object/whatever.

Or maybe I’m being r/whoosh’d with this one...

Nevermind, I didn’t read it closely enough. It’s for forward declaration of types that will eventually be properly defined. That’s a different animal entirely.

[fatcerberus]

🏠🚲

Given that exists implies existential types which these explicitly aren't, and that this is basically just an ambient declaration of a type, why couldn't the syntax simply be:

declare type Bar<T, U = number>;

declare type ManBearPig extends Man;
declare type ManBearPig extends Bear;
declare type ManBearPig extends Pig;

Both of the above declarations are syntax errors today, so we can just appropriate declare here and avoid the need for a new keyword entirely, right?

[dead-claudia]

@fatcerberus I like that idea of a declare type .... Doesn't create a new keyword, and it arguably is more appropriate to use here (and less of an abuse of terminology) than exists.

[rbuckton]

There's an issue with not being able to supply the minimum expected definition. Let's say I have the following package "packageA":

// tsconfig.json
{ "compilerOptions": { "target": "esnext", "lib": ["node"] } }

// index.ts
// yes, this is a bad example, but bear with me...
declare global {
  declare type Buffer;
}
export function copyBuffer(src: Buffer, dest: Buffer, srcStart: number, 
  destStart: number, count: number): void {
  while (count > 0) {
    dest.writeUint8(src.readUint8(srcStart++), destStart++);
    count--;
  }
}

Now lets say I consume "packageA" from "packageB":

// tsconfig.json
{ "compilerOptions": { "target": "esnext" } }

import { copyBuffer } from "packageA";

declare global {
  interface Buffer { iAmABuffer: boolean }
}

const src: Buffer = { iAmABuffer: true };
const dest: Buffer = { iAmABuffer: true };

copyBuffer(src, dest, 0, 0, 0);

The package "packageA" isn't indicating to "packageB" where Buffer should come from or what its definition needs to look like, so as far as "packageB" is concerned, it has properly satisfied the constraints of Buffer for "packageA".

By specifying a constraint (exists type Buffer extends { ... }), you can at least indicate the minimum implementation necessary. However, you could have just as easily written copyBuffer like this:

export function copyBuffer(
  src: { readUint8(offset: number): number },
  dest: { writeUint8(value: number, offset: number },
  srcStart: number,
  destStart: number,
  count: number): void { ... }

However this seems highly repetitive and overcomplicated.

At the end of the day, what this boils down to is this:

I have an API I produce that is optional and only usable in certain contexts. If you don't use this API, you don't need to also include the dependencies to satisfy that context.

So its not the case that just any Buffer will do, but rather "you can't use this particular API if you don't have the types for NodeJS's Buffer" and "if you don't use this particular API you don't need the types for NodeJS's Buffer".

In a way, I feel like this makes the whole exists type Buffer from "buffer" syntax more reliable, as it's not just any Buffer, but the one from that particular package/module. The syntax for requiring a global would be a bit trickier though. Perhaps what is needed is a package/lib hint for typings...

// just some syntax bikeshedding...
exists type Server from "net" in package "@types/node"; 
exists type Buffer in package "@types/node";
exists type Promise in lib "es2015.promises";

In these cases, your project wouldn't need to have a dependency on "@types/node" or a "lib": ["es2015.promises"] in your tsconfig.json if you don't use the APIs from the other package. If you don't have the requisite references, the types are basically never. If you do have the references, the types light up with the correct typings.

ExistentialTypeDeclaration:
  `exists` `type` Identifier TypeArguments? ExistentialFromClause? ExistentialInClause

ExistentialFromClause:
  `from` StringLiteral

ExistentialInClause:
  `in` `package` StringLiteral
  `in` `lib` StringLiteral

[fatcerberus]

I still don’t see why we can’t just use declare. It’s like every other use of declare we have already: to say that something exists for which you don’t (yet) have details about because the concrete definition is elsewhere.

[rbuckton]

I still don’t see why we can’t just use declare. It’s like every other use of declare we have already: to say that something exists for which you don’t (yet) have details about because the concrete definition is elsewhere.

Daniel explicitly calls out why just adding a declare or a module augmentation for the type is problematic in the Issues section, above. declare global { interface Buffer {} } "works" today, however that means you now have a Buffer type that is an empty object, so everything (except null/undefined/void) is assignable to the argument.

[fatcerberus]

Yes, I saw that bit. I specifically meant this:

declare type Buffer;
// instead of exists type Buffer;

Which is currently a syntax error.

[patrickroberts]

This feature reminds me of the weak symbol. To follow the established precedent, would weak work instead of exists?

[DanielRosenwasser]

@patrickroberts thanks for bringing up that example - I hadn't heard of weak symbols but it's conceptually very similar.

The issue with weak as a keyword is the confusion with another term we've used to describe certain types. Today, an object type which declares only optional properties and no signatures is considered weak, and we do extra checking in the presence of weak types.

[patrickroberts]

@DanielRosenwasser I can't say I'd heard that term describing such objects before. Is that term used just within TypeScript internals, or should that term also have meaning to end users of TypeScript?

[DanielRosenwasser]

It's not widely used, but it's been publicly explained enough, even within our release notes. I'm flexible, but I'd rather avoid weak unless we feel the other options don't suffice.

[HarryGifford]

This would be a really great addition to the language. I often want to use types and modules instead of classes, but the inability to hide the implementation of the type is annoying. Some examples where this would be useful in the wild are the file descriptor in Node's fs.open function and a lot of the WebGL types, such as WebGLBuffer.

Reason and Ocaml have abstract types that are used in exactly this way.

There's some more discussion in #321 which is for a similar request.

EDIT Never mind. I misread the proposal.

[jonaskello]

The exists keyword seem to cause some confusion. Since the explanation of this proposal mentions "forward declaration" in several places perhaps it could use a forward keyword instead of exists?

forward type Foo;

"Placeholder type" is also mentioned several times, so that might also be something to consider:

placeholder type Foo;

(I may be missing something, I only skimmed the proposal so I don't have a deep understanding of it yet)

[weswigham]

For syntax, I'm pretty fine with just

[declare] type Foo;

and

[declare] type Foo extends Whatever;

because it mirrors our other shorthand

declare module "foo";

in which we just take the part of the declaration we do have (the name) and elide the body.

[jonaskello]

Yes, having studied this proposal a bit deeper, declare makes a lot of sense since that is already used for the same purpose elsewhere.

[fatcerberus]

Yes, that’s what I’ve been saying all along! 😉

[AnyhowStep]

Just thought I'd post my declaration-merging-as-placeholder-type experiment here.

Playground

interface ExpectedConfigT {
  x : number,
  y : string,
  toString () : string,
}
/**
 * Your library
 */
interface ConfigT extends ExpectedConfigT {
  __doesNotExist? : unknown;
}
declare function getX () : ConfigT["x"];
declare function getToString () : ConfigT["toString"];
/**
 * Users of your library
 */
interface ConfigT {
  x : 1337,
  y : "hello",
  toString(encoding?: string, start?: number, end?: number): string,
}

/**
 * Type is `1337`
 */
const x = getX();

/**
 * Type is `(encoding?: string | undefined, start?: number | undefined, end?: number | undefined) => string`
 */
const toStringFunc = getToString();

[remcohaszing]

I had a very similar idea to solve the same problem.

What if, instead of checking if a type is declared globally, a type can be marked optional. If a type is marked optional, TypeScript will try to resolve this, but if it fails to do so, then it defaults to never. These types are declared locally only, so they don’t conflict with potential globals. Although they can be exported.

A practical trimmed down example based on postcss:

optional type Buffer = Buffer;

export default function postcss(css: string | Buffer): any;

If NodeJS types are includes, this is equal to:

type Buffer = Buffer;

export default function postcss(css: string | Buffer): any;

otherwise, this is equal to:

type Buffer = never;

export default function postcss(css: string | Buffer): any;

A practical trimmed down example based on axios:

optional type HTTPAgent = import('http').Agent;
optional type HTTPSAgent = import('https').Agent;

interface RequestConfig {
  httpAgent: httpAgent;
  httpsAgent: httpsAgent;
}

export default function request(config: RequestConfig): any;

If NodeJS types are includes, this is equal to:

type HTTPAgent = import('http').Agent;
type HTTPSAgent = import('https').Agent;

interface RequestConfig {
  httpAgent: httpAgent;
  httpsAgent: httpsAgent;
}

export default function request(config: RequestConfig): any;

otherwise, this is equal to:

type HTTPAgent = never;
type HTTPSAgent = never;

interface RequestConfig {
  httpAgent: httpAgent;
  httpsAgent: httpsAgent;
}

export default function request(config: RequestConfig): any;

---

At first I thought of using declare type, but I decided this is slightly different from declaring the type, so I went with optional type. (I want to share the idea, not nitpick about keywords.)

[thw0rted]

It's a bit of a hack, but you can sort of do your Agent example today. Try

// @ts-ignore
import type {Agent as HTTPAgent} from "http";
// @ts-ignore
import type {Agent as HTTPSAgent} from "https";

interface RequestConfig {
  httpAgent: any extends HTTPAgent ? never : HTTPAgent;
  httpsAgent: any extends HTTPSAgent ? never : HTTPSAgent;
}

The ts-ignore comment causes the type to resolve as any if http / https isn't available, then the conditional type converts it from any to never.

I believe you can actually do the same thing with your other examples if you just find the modules that @types/node declares and import them explicitly, e.g. import type {Buffer} from "buffer" -- the ts-ignore comment will still convert it to any without errors.

Maybe we just need a shorthand to do this without abusing ignore-comments? I don't know what the syntax would look like, but I'd love to have an import statement that says "give me type X from package Y if it's installed, otherwise FallbackType" (where I get to pick FallbackType).

ETA: You may find microsoft/TypeScript-DOM-lib-generator#1207 interesting, as right now I don't think there's a way to use this workaround with DOM types.

[thw0rted]

I keep getting feedback on this related SO question, and reading back over the thread here for more context. One thing I don't think has been discussed: does this proposal help when your code needs to return a placeholder type?

declare type Buffer {
  // ...?
}
declare type Socket {
  on(evt: "data", cb: (b:Buffer) => void): void;
}

export function doStuff(input: string): Promise<string>;
export function doStuff(input: Socket): Promise<Buffer>;
export function doStuff(input: string | Socket): Promise<string|Buffer> {
  if (isSocket(input)) {
    const bufferPromise = ...;
    input.on("data", ...);
    return bufferPromise;
  } else {
    return browserDoStuff(input);
  }
}

For isomorphic libraries, the idea is that the Socket and Buffer types would neatly erase out as never in a browser environment, while returning a Buffer type in a Node environment that is assignable to import type {Buffer} from "buffer". Would the returned placeholder Buffer assign correctly?

[DanielRosenwasser]

You're asking whether a local placeholder with the name Buffer would be compatible with the real Buffer if it was imported, right?

In that case, no, and that was part of the concern I brought up over in #31894 (comment)

One problem with placeholders is that they aren't able to discuss the values and types of a module that might only partially exist. While this isn't widespread in the module ecosystem right now, that ecosystem is fairly nascent and we don't want to box ourselves into a design that doesn't have a design ready. But we did ask ourselves quite a few times whether it made sense to have a

placeholder module "foo" {
  export placeholder type Thing;
}

and we want to understand if there's something more there.

Last I checked, Buffer was declared globally, but ideally a solution like placeholders would allow a user to also declare a placeholder module too.

[thw0rted]

Thanks, Daniel. Just to be clear, is the answer different for imported versus global types? Like, if I build my library without dom in my lib, and had a placeholder Element, then the consumer used my library and included dom, the Element I return would be the real global top level type?