Working with node and only esm packages

[alonronin]

Current Behavior

import { execa } from 'execa';

execa('echo', ['test']).then(console.log);

this cause an error

/Users/user/www//env-vars/dist/apps/exec/main.js:8
module.exports = require("execa");
                 ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/user/www/env-vars/node_modules/execa/index.js from /Users/user/www/env-vars/dist/apps/exec/main.js not supported.
Instead change the require of index.js in /Users/user/www/env-vars/dist/apps/exec/main.js to a dynamic import() which is available in all CommonJS modules.
    at Object.execa (/Users/user/www/env-vars/dist/apps/exec/main.js:8:18)
    at __webpack_require__ (/Users/user/www/env-vars/dist/apps/exec/main.js:32:41)
    at /Users/user/www/env-vars/dist/apps/exec/main.js:45:17
    at /Users/user/www/env-vars/dist/apps/exec/main.js:48:3
    at Object.<anonymous> (/Users/user/www/env-vars/dist/apps/exec/main.js:53:12)

Expected Behavior

The package in the example is execa, however it happens with any esm only package as node-fetch.

Environment

Node : 16.13.0
  OS   : darwin x64
  yarn : 1.22.17
  
  nx : 13.2.2
  @nrwl/angular : undefined
  @nrwl/cli : 13.2.2
  @nrwl/cypress : undefined
  @nrwl/devkit : 13.2.2
  @nrwl/eslint-plugin-nx : undefined
  @nrwl/express : undefined
  @nrwl/jest : 13.2.2
  @nrwl/linter : 13.2.2
  @nrwl/nest : undefined
  @nrwl/next : undefined
  @nrwl/node : 13.2.2
  @nrwl/nx-cloud : undefined
  @nrwl/react : undefined
  @nrwl/react-native : undefined
  @nrwl/schematics : undefined
  @nrwl/tao : 13.2.2
  @nrwl/web : undefined
  @nrwl/workspace : 13.2.2
  @nrwl/storybook : undefined
  @nrwl/gatsby : undefined
  typescript : 4.4.4
  rxjs : 6.6.7
  ---------------------------------------
  Community plugins:

[nartc]

Hi @alonronin , can I have more information? Or a repo to replicate this? I'm seeing the issue with ESM and want to figure out the steps to get to the error.

[alonronin]

@nartc sure, here is an example repo:
https://github.com/alonronin/esm-nx-example

$ git clone && yarn && yarn nx run execa:serve

[dobromyslov]

Yesterday bumped into this error when migrated my NX node project from CommonJS to ESM.

Root cause investigation:

1. @nrwl/node:build calls getNodeWebpackConfig()
2. getNodeWebpackConfig() calls getNodePartial()
3. getNodePartial() imports 'webpack-node-externals' plugin as nodeExternals and calls it without configuration options.
4. Then look at the webpack-node-externals options.importType. It's default value is 'commonjs' thats why it tells webpack to load all node externals using require() instead of ESM module import.

Workaround:

1. npm install --save-dev webpack-node-externals
2. Create webpack.config.cjs in the application directory:

// @filename ./apps/myapp/webpack.config.cjs
const nodeExternals = require('webpack-node-externals');
module.exports = (config, context) => {
  return {
    ...config,
    externalsPresets: {
      node: true
    },
    output: {
      ...config.output,
      module: true,
      libraryTarget: 'module',
      chunkFormat: 'module',
      library: {
        type: 'module'
      },
      environment: {
        module: true
      }
    },
    experiments: {
      outputModule: true,
    },
    externals: nodeExternals({
      importType: 'module'
    })
  }
}

3. Change your angular.json or workspace.json

...
"myapp": {
  ...
  "architect": {
    ...
    "build": {
      "builder": "@nrwl/node:build",
      "options": {
        ...
        "webpackConfig": "apps/myapp/webpack.config.cjs",
      },
    },
  },
}

4. Don't forget to add "type": "module" to your package.json to tell the Node that your project uses ESM instead of default CommonJS. Also you may place package.json with this one line to the application folder if you want to move only one app to ESM.
5. Slightly change decorate-angular-cli.js

/*
const fs = require('fs');
const os = require('os');
const cp = require('child_process');
*/
import fs from 'fs';
import os from 'os';
import cp from 'child_process';

6. Use Node 16+. (I tested with v14 but got some errors in some cases like it can't find modules with .ts extension and requires only .js extension or it can't resolve directories as modules. Issues disappeared after migration to Node v16.13).
7. Make sure you are using latest Webpack (I tested with "webpack": "^5.65.0", "@types/webpack": "5.28.0", "webpack-node-externals": "^3.0.0")
8. And make sure your tsconfig.base.json has this option enabled:

...
  "compilerOptions": {
    "module": "ES2020",
  }

After these steps are done your app will be assembled by Webpack with externals imported using ESM way. And you'll see in dist your external dependencies like this:

import * as __WEBPACK_EXTERNAL_MODULE__google_cloud_bigquery_18c5c905__ from "@google-cloud/bigquery";
//...
/***/ "@google-cloud/bigquery":
/*!*****************************************!*\
  !*** external "@google-cloud/bigquery" ***!
  \*****************************************/
/***/ ((module) => {

var x = y => { var x = {}; __webpack_require__.d(x, y); return x; }
var y = x => () => x
module.exports = __WEBPACK_EXTERNAL_MODULE__google_cloud_bigquery_18c5c905__;

/***/ }),

[dobromyslov]

@nartc I propose a feature request: add --esm configuration flag to the @nrwl/node:build which allows us to build node with

nodeExternals({
  modulesDir,
  importType: 'module'
})

nx/packages/node/src/utils/node.config.ts

Line 27 in 0b3cab0

webpackConfig.externals = [nodeExternals({ modulesDir })];

And also this flag should tune the default webpack config used during build as described in the comment above

[alonronin]

@dobromyslov thanks! would be great if this is on nx roadmap to fix node's generators.

[github-actions]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[keverw]

I think support is still needed? I haven't gave them a try recently.

[Bielik20]

I have experimented a little with @nrwl/js module. I created a generator that adjusts @nrwl/js:lib to EMS format.

There are some caveats though:

1. You still need to manually add .js extension to your files. I could find a lint rule that does that automatically with typescript files.
2. You need to run jest unit tests with experimental flag like so: NODE_OPTIONS=--experimental-vm-modules nx run foo:test
3. jest handles EMS modules BUT you cannot use extension in imports inside spec files. I do not know why, maybe I am missing something here.

It would certainly be possible to adjust @nrwl/js module to allow ESM libs but I think some more work needs to be added. Mainly to address the issues I mentioned above, and of course app builders should be able to handle pure (ESM) modules.

[github-actions]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[fbartho]

Still real pain

[github-actions]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[fbartho]

Fighting the stalebot!

[stomvi]

Will #10414 fix this?

[github-actions]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

[itPiligrim]

Up

[hstanford]

I was running into a bunch of issues trying to run plain typescript that targets esm and also uses other libraries in my workspace. I got a custom module loader working that hooks in the commonjs path resolver (so directory imports work and you don't need the contentious '.js' suffix!) and also grabs the library paths from the base tsconfig so it knows where to find your libraries.

Easy running of ts->esm scripts e.g.

TS_NODE_PROJECT=packages/test/tsconfig.lib.json  NX_BASE_TSCONFIG=./tsconfig.base.json node --loader nx-ts-esm-loader packages/test/index.ts

https://www.npmjs.com/package/nx-ts-esm-loader - hope it helps!

[thomasskk]

up

[alljinx]

up

[github-actions]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.