Merging UMD module definitions exporting to same Global Identifier (E.g. D3 version 4)

[tomwanzek]

TypeScript Version: 2.0.0

@mhegazy As per your suggestion under the already closed issue #7125 , I am opening a new issue for
the following. (I am simply copying over my outlines and references as per yesterday's post.)

I have a quick question regarding the following quote at the very end of the Solution summary by @RyanCavanaugh :

These declarations may engage in module merging, though this should probably be discouraged somehow.

I have been drawing up new typescript definitions for Mike Bostock's popular, newly modularized version 4 of the D3 data visualization tools. D3 is now split up into several modules, e. g. d3-selection, d3-transition, d3-shape etc...

There is also a standard pre-built bundle of the 'core' modules, which is provided as a single module d3.
The modules are structured as UMD modules for bundled/unbundled use in vanilla script as well as module import use cases.

In writing the definitions, I came across the following issue related to their UMD character as. For the vanilla script scenario:

1. if the standard pre-built bundle of modules is loaded from the single bundle file, it exposes a d3 global with the objects exposed by each of the modules which feed the standard bundle
2. if the scripts of the d3 modules are individually included (i.e. unbundled), each of them exports to/extends the same d3 global. Mike's intent being ease of code reuse for D3 users in the vanilla scenario.

Ad 1): Creating a bundle definition with UMD characteristics for the default bundle d3 is as simple as re-exporting the relevant 'core' modules and adding the export as namespace d3; for the global.

Ad 2): I am running into the issue that, adding the export as namespace d3; to the individual modules, e.g. d3-selection, d3-transition etc., creates a duplicate identifier compile error for the d3 identifier. (Typescript v2.0.0.) (Note: there are no identifier conflicts between the objects exported from the individual modules)

Despite the aforementioned quote, I suspect this is expected compiler behavior? Is there a preferred way to accomplish the module merging into the global d3?

Code

I created a repo here to stage the D3 definitions while I am drafting them. I did not use my DefinitelyTyped fork to create a pull-request right away, because I am using this typing of callback function contexts, which was not yet available in typescript 1.8.10.

I intend to move them into DefinitelyTyped as soon as possible for those that have completed 'shape tests' of the definitions. And then incrementally as testing completes.

The repo itself carries an issue for the d3 global question raised above here. Representative definition files can be found here for e. g.:

• d3-selection,
• d3-transition and
• d3-shape

Note, that these definitions currently do not individually have the export namespace as d3;, because of the mentioned issue. I added them locally and get the compile error.

There is a definition file for the 'bundled' d3 module here, which uses re-exports and has the global export.

Expected behavior:

Multiple declaration files for UMD modules should be allowed to export to the same global identifier, e.g. d3 in the present case, using export as namespace d3; at the constituent module level.
Of course, this is subject to, there not being any identifier naming conflicts between the exported objects which are to be merged into the global.

While, in general, it may be preferable to avoid module merges into the same global, there seems to be a valid rationale in circumstances such as D3.

Actual behavior:

The compiler throws a duplicate identifier 'd3' error in the individual modules containing the export as namespace d3; statement.

[tomwanzek]

Just wondering if anyone has a chance to comment on the above issue.

@mhegazy, as per your request this issue was opened as new. I know you are all very busy on Typescript. (And thanks a lot for all the hard work on it!)

Work on the definitions for most of the separate D3 modules is finalized (including tests). There is a smaller remainder set to be done. I started to get ready preparing a pull request for DefinitelyTyped/types-2.0 in my fork, based on the completed work in the above repo.

This issue is one of the somewhat fundamental items still open to mimick the D3 UMD + bundle/unbundle behavior.

So I would appreciate your insight on the above.

[tomwanzek]

If you prefer, here is the pull request for DefinitelyTyped/types-2.0, which is based on the aforementioned repo:

D3 Version 4.1.x modules first batch

[tomwanzek]

With the exception of one pending module, all D3 standard bundle modules have now been merged into the DefinitelyTyped/types-2.0 branch PR 10228 or have been submitted for a merge PR 10435.

With the exception of the d3 standard bundle definition itself. Each of the modules is affected by this issue.

[tomwanzek]

@RyanCavanaugh I would be most indebted for some insight related to this issue. Thanks in advance.

[DanielRosenwasser]

Wow I'm actually really sorry that this never got triaged. It has been busy given that people have been on vacation. Reading through now.

[DanielRosenwasser]

@tomwanzek so as modules, d3 just re-exports its smaller components, but as scripts, it seems like each script augments the global d3 object.

I'm not sure if UMD style declarations can accurately model the granularity that's taking place here.

Some ideas I brainstormed on (that I don't think work) are:

• d3 simply performs an import of d3-array (with no new named entities - import "../d3-array";)
• d3 also has an export as namespace d3 for its global declaration.
• d3-array uses a namespace declaration and an export = declaration of that namespace.
• d3-array also performs a module augmentation of d3 with the same members.

The problem is that module augmentations can't add new entities to the top-level declaration of a module if I recall correctly. @RyanCavanaugh any insights here?

[tomwanzek]

@DanielRosenwasser thanks a lot for chiming in. I now all of you are very busy making TypeScript even more awesome.

I will add on to this thread later today by providing usage examples by scenario. Unfortunately, I have to rush elsewhere right now...

[RyanCavanaugh]

The problem is that module augmentations can't add new entities to the top-level declaration of a module

We've removed this restriction.

The proposal to merge UMD globals is somewhat at odds with the problem described at #9771 - there, we want the "first" UMD global to win.

I honestly can't think of a good way to represent this pattern in a way that works if you have both a plugin and a "full" d3 loaded at the same time.

[tomwanzek]

Apologies for the slight delay, I was off the grid for a little while longer than I had hoped for.

I'll try to sum up the key scenarios of how the new D3 modules can be used and highlight where I think this issue kicks in right now [i.e. Scenario (4)]. Below I am assuming they have been installed from @types. You can think of the scenarios as a mutually exclusive project-by-project choice. They would not be combined in a single project

Scenario (1) Module Imports of Separate Modules

Standard module imports of only the required modules for the given project. The individual modules have several named exports, they have no default export and there is no d3 object/namespace exported. (No global is used in this scenario.)

// use-separate.ts
import * as d3Select from 'd3-selection';
import * as d3Transition from 'd3-transition';
import { symbol } from 'd3-shape';

let b = d3Select.select('body');
let t = d3Transition.transition('newTransition');
let sym = symbol();

This scenario is covered by the current definitions that have just been merged to DefinitelyTyped/types-2.0 and have been published to @types.

Scenario (2) Module Import of Standard D3 Bundle

Mike Bostock, author of D3, publishes a "pre-build standard bundle" of key D3 modules. This bundle is published as the d3 package and can be imported as a module, which re-exports the named exports of the D3 modules included in the bundle. (No global is used in this scenario.)

The use case with module import of the d3 bundle simply looks like this:

// use-standard-bundle.ts
import * as d3 from 'd3';

let b = d3.select('body');
let t = d3.transition('newTransition');
let sym = d3.symbol();

As usual the use of d3 here is simply an import alias, any other name could be used to alias.

In principle, this scenario is supported. I have created a UMD module definition file for *d3, which is included in the DefinitelyTyped PR 10453 for types-2.0 and covers this scenario. It can be found here, and structurally looks like this:

// ...
export as namespace d3;

export * from 'd3-array';
export * from 'd3-axis';
// ...
export * from 'd3-zoom';

The primary reason why this PR has not been able to be merged is because I had posed some questions related to treatment of the legacy D3 version 3.5.17 definition, which is still used by a few other libraries/definitions in DefinitelyTyped.

Scenario (3) Vanilla Script using Standard D3 Bundle

The standard D3 bundle Mike Bostock publishes for convenience as mentioned in Scenario (2) can also be used as a vanilla script.

<!-- index.html -->
<script src="https://d3js.org/d3.v4.js"></script>

In this scenario, D3 exposes a d3 global.

So, consumption would look like this:

// vanilla-script-using-bundle.ts

/// <reference types = 'd3' />

let b = d3.select('body');
let t = d3.transition('newTransition');
let sym = d3.symbol();

Since the definition file for the d3 bundle mentioned in scenario (2) is written as a UMD module definition, this scenario is covered. Subject to merge and publication. The d3 global is exposed through the export as namespace d3;

Scenario (4) Vanilla Script Use of Individual D3 Modules

This scenario is where the issue currently kicks in. Each individual d3 module can also be used as a vanilla script:

<!-- index.html -->
<script src="https://d3js.org/d3-selection.v1.js"></script>
<script src="https://d3js.org/d3-transition.v1.js"></script>
<script src="https://d3js.org/d3-shape.v1.js"></script>

Each of these script exposes its respective exports through the d3 global, i.e. they merge without identifier conflicts.

So, as under scenario (3) consumption should look like this:

// vanilla-script-using-individual-scripts.ts

/// <reference types = 'd3-selection' />
/// <reference types = 'd3-transition' />
/// <reference types = 'd3-shape' />

let b = d3.select('body');
let t = d3.transition('newTransition');
let sym = d3.symbol();

This scenario is currently not supportable by simply adding export as namespace d3; to the individual D3 module definition files (e.g. d3-selection, d3-transition etc.)

[RyanCavanaugh]

I think the best overall solution here is to just not use the built-in UMD support. The only real downside is that you won't get "proper" versioning between types versions, but for a library like d3 you're very unlikely to be pulling disparate type versions. Instead, just have the libraries augment the global scope using declare global

[tomwanzek]

@RyanCavanaugh thanks for all the hard work on TS2.

As for your above comment, could you elaborate a bit.

If I understand your proposal correctly, it would imply that every D3 module level definition file would have two parts:

• a complete proper module section providing all the exports for import based scenarios
• a declare global section which completely duplicates all the exports (including ultimately JSDoc comments)

The declare global augmentation also appears to raise the following question: Assuming that, the global object d3 in Scenario 4 adheres to an interface D3. Theoretically, D3 could be augmented using declare global. However, where should the d3 variable be declared? No single constituent module of the D3 standard bundle "owns" the variable declaration itself. In principle, there is no primus inter pari D3 script that will always be loaded in Scenario 4.

Should I be wrong about the duplication requirements, I apologize in advance for having missed something. If, however, said duplication is required, this seems rather prohibitive from a maintainability perspective.

[tomwanzek]

Is there any update on this consideration? It is still a gap in the definitions for D3. Thx, as always.

[RyanCavanaugh]

I don't think there's any way we're going to be able to support this scenario directly. This is just going to have to be split up into separate module and global definitions.