Publish CJS and ES transpiled files in different versions

[trivikr]

Is your feature request related to a problem? Please describe.

Each modular package on npm ships with both CJS and ES transpiles files.

$ npm install @aws-sdk/client-s3

$ du -sh node_modules/@aws-sdk/client-s3 
6.1M    node_modules/@aws-sdk/client-s3

$ du -sh node_modules/@aws-sdk/client-s3/dist-cjs
1.6M    node_modules/@aws-sdk/client-s3/dist-cjs

$ du -sh node_modules/@aws-sdk/client-s3/dist-es 
1.3M    node_modules/@aws-sdk/client-s3/dist-es

$ cat node_modules/@aws-sdk/client-s3/package.json| grep '"main"'
  "main": "./dist-cjs/index.js",

$ cat node_modules/@aws-sdk/client-s3/package.json| grep '"module"'
  "module": "./dist-es/index.js",

Since dist-cjs is used in Node.js and dist-es is consumed by browser, they can be shipped into two different packages/versions.

Describe the solution you'd like

Ship browser and Node.js versions in different package names (say with suffix -node and -browser)

This suggestion was followed in preview versions of AWS SDK for JavaScript (v3).
For example: @aws-sdk/client-s3-node

It was discontinued primarily because it's not scalable.

• It generates duplicate code which needs to be maintained in repository.
• Supporting new environment (say ReactNative) would require adding new package per client (say with suffix -rn).
• Supporting new module module system (say ESM) would require adding new package per client (say node-esm)

Ship different dist in prerelease tags

This suggestion includes using prerelease tags for shipping specific versions where reducing npm install size is important. For example:

• Ship all dist-* in version X.Y.Z
• Ship only dist-node-cjs in version X.Y.Z-node-cjs
• Ship only dist-node-esm in version X.Y.Z-node-esm
• Ship only dist-node-types in version X.Y.Z-node-types

The separate tags are not required for browser and react-native environments, as they are expected to use bundlers and are not dependent on npm install size.

In environments where npm install size is not a concern, developers can use X.Y.Z version as follows:

// package.json
{
  "dependencies": {
    //...
    "@aws-sdk/client-s3": "X.Y.Z",
    //...
  }
}

In environments where npm install size is a concern (while using ESM on Node.js), developers can use X.Y.Z-node-esm version as follows:

// package.json
{
  "dependencies": {
    //...
    "@aws-sdk/client-s3": "X.Y.Z-node-esm",
    //...
  }
}

For developers requiring types for while using prerelease versions, then can just add X.Y.Z-node-esm in devDependencies as follows:

// package.json
{
  "dependencies": {
    //...
    "@aws-sdk/client-s3": "X.Y.Z-node-esm",
    //...
  },
  "devDependencies": {
    //...
    "@aws-sdk/client-s3": "X.Y.Z-types",
    //...
  }
}

The different versions will be managed in the repo by creating custom package.json files for each prerelease tag, similar to how it's done in tsconfig. The release script will use custom package.json when publishing packages on npm.

• package.json: will include all configurations.
• package.node.cjs.json: will include configurations and dependencies only for CommonJS.
• package.node.esm.json: will include configurations and dependencies only for ESM.
• package.types.json: will include configurations and dependencies only for types.

Describe alternatives you've considered

Scrapping of dist-* folders which are not being used in postinstall script.

Additional context

Request proposed earlier in:

• Reduce duplicate code to halve large package sizes #1610
• Wasteful Package Size #2132
• esm build targets es5 #2056

[trivikr]

Removing dist-types and dist-es from npm publish will reduce the publish size further.

For @aws-sdk/client-s3, the npm publish size reduction would be 75% as follows:

$ npm install @aws-sdk/[email protected]
...

$ du -sh --apparent-size node_modules/@aws-sdk/client-s3
5.2M    node_modules/@aws-sdk/client-s3

$ rm -rf node_modules/@aws-sdk/client-s3/dist-es node_modules/@aws-sdk/client-s3/dist-types
1.4M    node_modules/@aws-sdk/client-s3

[seriousme]

FYI: node-fetch uses a different approach

node-fetch is an ESM-only module - you are not able to import it with require. We recommend you stay on v2 which is built with CommonJS unless you use ESM yourself. We will continue to publish critical bug fixes for it.

Alternatively, you can use the async import() function from CommonJS to load node-fetch asynchronously:

// mod.cjs
const fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));

https://github.com/node-fetch/node-fetch#loading-and-configuring-the-module

Kind regards,
Hans

[trivikr]

Revisiting this idea: looking at semver 2.0 specifications, it would be more helpful to use build metadata instead of prerelease tags.

For example:

• Ship all dist-* in version X.Y.Z
• Ship only dist-node-cjs in version X.Y.Z+node-cjs
• Ship only dist-node-esm in version X.Y.Z+node-esm
• Ship only dist-node-types in version X.Y.Z+node-types

[trivikr]

Revisiting this idea: npm does not support publishing versions with build metadata as described in npm/npm#12825

We will need to use prerelease tags, as shown in example below:

• Ship all dist-* in version X.Y.Z
• Ship only dist-cjs in version X.Y.Z-latest.cjs
• Ship only dist-esm in version X.Y.Z-latest.esm
• Ship only dist-types in version X.Y.Z-latest.types
• Ship only dist-web in version X.Y.Z-latest.web

The prefix latest. is added to prerelease tags do that it's not confused with other prerelease tags like alpha, beta, rc etc.

[simonbuchan]

@trivikr bikeshedding, but I'd prefer release. to latest. - latest has meaning as a tag.

[trivikr]

I'd prefer release. to latest. - latest has meaning as a tag.

We'd thought of using latest prefix, just to indicate it's an extension of latest tag.
But we're now inclined towards providing a CLI to filter artifacts, say aws-sdk-js-cli filter-artifacts --variants node.cjs

I'll provide an update here, once the design is internally reviewed.

[simonbuchan]

Huh, I dunno how well that would integrate with the various package managers out there, but I'm happy to wait and see. The package size thing isn't really an issue for me, because I bundle everything, and they deal quite well, especially with exports.

[trivikr]

Revisiting. We only need Node.js specific variant to reduce install size.
The browser specific variant is not needed, as most users use bundlers.

[github-actions]

Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.