Not working with native ES modules #59

cullylarson · 2019-05-21T03:56:35Z

I'm moving a project over to using node's native ES modules (enabled with the --experimental-modules flag). After updating my code, module-alias is no longer working. I tried adding this to the root of my app (this is the method I was using before transitioning to esm):

require('module-alias/register')

I tried changing it to:

import 'module-alias/register'

I tried requiring when starting the server:

node --experimental-modules -r module-alias/register server/app.js

The first aliased import in my app is this:

import {responseError} from '@app/lib/response'

I'm getting this error from it:

internal/modules/esm/default_resolve.js:69
  let url = moduleWrapResolve(specifier, parentURL);
            ^

Error: Cannot find package '@app/lib' imported from server/app.js
    at Loader.resolve [as _resolve] (internal/modules/esm/default_resolve.js:69:13)
    at Loader.resolve (internal/modules/esm/loader.js:70:33)
    at Loader.getModuleJob (internal/modules/esm/loader.js:143:40)
    at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:43:40)
    at link (internal/modules/esm/module_job.js:42:36)

The relevant lines in my package.json are:

"_moduleAliases": {
    "@app": "./server"
},

I'm starting the app like this:

node --experimental-modules server/app.js

module-alias worked fine using CommonJS. The only change I made to the code was changing requires to imports.

The text was updated successfully, but these errors were encountered:

jdt3969 · 2019-06-10T16:20:30Z

@cullylarson Ran into the same issue.

It appears that the new esm code isn't running the _resolveFilename which is the core of this library. Based on the docs it looks as though they are moving off of this library's hack and onto a standard feature: https://github.com/nodejs/node/blob/master/doc/api/esm.md#experimental-loader-hooks It's still experimental though.

I rewrote and reduced a lot based on known things within my library but this code is working for me:

// custom-loader.mjs
import path from 'path';
import npmPackage from './package.json';

const getAliases = () => {

  const base = process.cwd();

  const aliases = npmPackage.aliases || {};

  const absoluteAliases = Object.keys(aliases).reduce((acc, key) =>
    aliases[key][0] === '/'
      ? acc
      : { ...acc, [key]: path.join(base, aliases[key]) },
    aliases)

  return absoluteAliases;

}

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
    && (path.length === alias.length || path[alias.length] === '/')
}

const aliases = getAliases();

export const resolve = (specifier, parentModuleURL, defaultResolve) => {
  
  const alias = Object.keys(aliases).find((key) => isAliasInSpecifier(specifier, key));

  const newSpecifier = alias === undefined
    ? specifier
    : path.join(aliases[alias], specifier.substr(alias.length));

  return defaultResolve(newSpecifier, parentModuleURL);
}

Then:
node --no-warnings --experimental-modules --es-module-specifier-resolution=node --loader ./custom-loader.mjs index.js

cullylarson · 2019-06-10T19:31:00Z

@jdt3969 Thanks for sharing. I'll give it a try on my next node project.

Kehrlann · 2019-06-10T19:56:47Z

Very interesting ! So if I understand correctly, you must just provide a .js file in a --loader flag ; and that .js file be a module that exports a resolve function, which basically does what module-alias does. (There might be a chicken and egg problem if you want to use module-alias programmatically though)

So it'd be very easy to do a pull request (wink wink, nudge nudge) that exports a neat little module that wraps module-aliases' resolve function, which can then be used like (rough idea, semantics TBD) :

node --no-warnings --experimental-modules --es-module-specifier-resolution=node --loader ./node_modules/module-alias/es6-loader.js index.js

@cullylarson @jdt3969 interested in doing a PR ?

@ilearnio any additional thoughts ?

cullylarson · 2019-06-10T23:23:13Z

@Kehrlann Thanks for the kind invitation to do a PR. I really appreciate the way you presented it. I'm near a deadline on a project right now and about to start another, otherwise I would take you up on the offer.

kirkouimet · 2019-11-27T18:02:46Z

You guys think we can do this without using the --loader flag and instead do it programmatically? Now that Module._resolveFilename isn't being used for import statements I wonder what is. May dive into it this week.

eouia · 2020-09-01T08:30:33Z

Not yet solved natively without execution flags?

JakobJingleheimer · 2021-01-05T22:43:19Z

@kirkouimet @eouia the node ems documentation linked above suggests no, the flag is required.

JakobJingleheimer · 2021-01-05T23:20:25Z

@jdt3969's above solution worked for me after I realised it was looking for a property in package.json called aliases instead of module-alias's documented _moduleAliases (I prefer jdt3969's aliases).

I tidied up the above code and added some pre-computing of values that can't change during execution: gist

EDIT: ~~⚠️ This appears to have broken somewhere between node v15.1.0 and v15.6.0 (jdt3969's too)~~

This was due to an error in my own source-code, which node ESM erroneously reported as coming from alias-loader.mjs

TimDaub · 2021-01-15T13:50:30Z

This is really interesting and the gist looks promising @jshado1.

But I'm wondering if this functionality can already be used when publishing a module to npm. I guess not, because how am I supposed to specify --experimental-loader=./alias-loader.mjs ./index.mjs as a user of said module?

JakobJingleheimer · 2021-01-15T20:04:00Z

@TimDaub Thanks!

But I'm wondering if this functionality can already be used when publishing a module to npm.

You probably shouldn't as loaders are imminently changing.

how am I supposed to specify --experimental-loader=./alias-loader.mjs ./index.mjs as a user of said module?

I'm not sure I understand what you mean—are you talking about where your module would itself depends on the loader, or someone consuming your module would need to consume it via --experimental-loader? For both, the answer would be roughly the same: as loaders currently stand in their experimental form, the consuming user would need to manually add it to their command (ex in package.json's "scripts") regardless of "who" is using it.

jaschaio · 2021-03-31T12:18:46Z

@jdt3969's above solution worked for me after I realised it was looking for a property in package.json called aliases instead of module-alias's documented _moduleAliases (I prefer jdt3969's aliases).

Worked for me as well, though it's important to note that you now need to use --experimental-loader=./custom-loader.mjs. I added as well --experimental-json-modules so that it works with .json files.

TimDaub · 2021-03-31T12:22:24Z

I'm not sure I understand what you mean—are you talking about where your module would itself depends on the loader, or someone consuming your module would need to consume it via --experimental-loader?

Later. Imagine I published a package that relied on starting node with --experimental-loader but then I'd also publish it to npm for others to use it. Now they'd have to be aware that now their project too as to be started with --experimental-loader.

the consuming user would need to manually add it to their command (ex in package.json's "scripts") regardless of "who" is using it.

That makes it unusable for npm packages until a version of node is out that removes the flag.

szydlovski · 2021-09-21T18:04:51Z

I understand that this is kind of an issue with Node itself, but it would be nice if the README included a warning about this package not being compatible with native ES modules. I just spent an hour questioning my sanity only to find out that the feature I've been hopelessly debugging was never intended to work at all.

Kehrlann · 2021-09-22T07:48:10Z

@szydlovski good point, would you want to submit a PR?

szydlovski · 2021-09-22T08:07:57Z

@Kehrlann Sure, here's a proposal

JakobJingleheimer · 2021-10-23T11:40:40Z

@TimDaub "unusable" does not seem accurate at all, and such CLI flags are quite common in many, many libraries. Support for a non-CLI option has been briefly discussed as a possibility in future, but if it happens, it would likely be quite a bit down the road. How loaders in Node.js will (almost surely) work once loader chaining is supported would be something like

$> node --loader https-loader --loader typescript-loader ./your-app.ts

TimDaub · 2021-10-23T12:46:19Z

Having to mention in the installation instruction on the library level that the using application should be initiated with a loader flag makes it unusabe.

It's fragile declaring this information in a readme. On the library level, the loader dependency should be described in the package.json and any using higher level package should interpret the lower libraries' loaders through the package.json.

But I guess that's out of scope for module-alias.

JakobJingleheimer · 2021-10-23T17:25:58Z

That sounds a bit dramatic. It's called a README (in screaming capitals) for a reason…. This is classic RTFM.

Module alias is obsolete now anyway with import mapping.

TheDirigible · 2022-05-10T01:56:07Z

This problem has been plaguing me for years because I want to write good quality code that is shared between node & browser.
I finally found a system that works:

Place 'nesm.js' in the root of your project
[optional] Place the 'nesm' shell script in your path, make it executable
Run scripts with: 'nesm file_to_run.js' or 'node path/to/nesm.js -- file_to_run.js'

'nesm.js'

/*
 * esm and module-alias do not play nicely together.
 * this precise arrangement is the only way I found to make it work.
 * you can run this from anywhere in your project hierarchy.
 * you can use args, and use in npm scripts.
 * encourage the node.js devs to make this work natively.  ux matters.
 * ---- CAVEATS
 * will not work with "type":"module"
 * ---- SETUP
 * place 'nesm.js' in the root of your project
 * [optional] place the 'nesm' shell script in your path, make it executable
 * ---- USAGE
 * > nesm file_to_run.js
 * to run without the nesm shell script:
 * > node path/to/nesm.js -- file_to_run.js
 * to run with nodemon:
 * > nodemon -- path/to/nesm.js -- file_to_run.js
*/
require = require('esm')(module);   // eslint-disable-line no-global-assign
require('module-alias/register');   // must come after esm for some reason

let runNext;
for(const arg of process.argv) {
    if(runNext) {
        let filename = arg;
        if(filename[0]!='.' && filename[0]!='/') filename = './'+filename;
        require(filename);
        break;
    }
    runNext = (arg=='--');
}

'nesm' shell script

#!/bin/bash
if [ -z $1 ]; then
    echo "Node esm runner.  Usage: nesm file_to_run.js"
    exit 1
fi
baseDir=$( pwd )
while [ ! -f "$baseDir/nesm.js" ]; do
    if [ ${#baseDir} -le 1 ]; then
        echo "nesm.js not found in folder ancestry"
        exit 1
    fi
    baseDir="$(dirname "$baseDir")"
done
file1=$(realpath $1);
node $baseDir/nesm.js -- $file1

euberdeveloper · 2022-07-18T22:14:08Z

@cullylarson Ran into the same issue.

It appears that the new esm code isn't running the _resolveFilename which is the core of this library. Based on the docs it looks as though they are moving off of this library's hack and onto a standard feature: https://github.com/nodejs/node/blob/master/doc/api/esm.md#experimental-loader-hooks It's still experimental though.

I rewrote and reduced a lot based on known things within my library but this code is working for me:
// custom-loader.mjs
import path from 'path';
import npmPackage from './package.json';

const getAliases = () => {

  const base = process.cwd();

  const aliases = npmPackage.aliases || {};

  const absoluteAliases = Object.keys(aliases).reduce((acc, key) =>
    aliases[key][0] === '/'
      ? acc
      : { ...acc, [key]: path.join(base, aliases[key]) },
    aliases)

  return absoluteAliases;

}

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
    && (path.length === alias.length || path[alias.length] === '/')
}

const aliases = getAliases();

export const resolve = (specifier, parentModuleURL, defaultResolve) => {
  
  const alias = Object.keys(aliases).find((key) => isAliasInSpecifier(specifier, key));

  const newSpecifier = alias === undefined
    ? specifier
    : path.join(aliases[alias], specifier.substr(alias.length));

  return defaultResolve(newSpecifier, parentModuleURL);
}
Then: node --no-warnings --experimental-modules --es-module-specifier-resolution=node --loader ./custom-loader.mjs index.js

You should have said that in package.json you expected the key aliases and no more _moduleAliases...

JakobJingleheimer · 2022-07-19T10:06:17Z

You should have said that in package.json you expected the key aliases and no more _moduleAliases...

@euberdeveloper I did #59 (comment) you can also see it on line 9 of the code. You're welcome btw.

euberdeveloper · 2022-11-13T18:42:05Z

In this code there is a problem with Windows paths.

@cullylarson Ran into the same issue.
It appears that the new esm code isn't running the _resolveFilename which is the core of this library. Based on the docs it looks as though they are moving off of this library's hack and onto a standard feature: https://github.com/nodejs/node/blob/master/doc/api/esm.md#experimental-loader-hooks It's still experimental though.
I rewrote and reduced a lot based on known things within my library but this code is working for me:
// custom-loader.mjs
import path from 'path';
import npmPackage from './package.json';

const getAliases = () => {

  const base = process.cwd();

  const aliases = npmPackage.aliases || {};

  const absoluteAliases = Object.keys(aliases).reduce((acc, key) =>
    aliases[key][0] === '/'
      ? acc
      : { ...acc, [key]: path.join(base, aliases[key]) },
    aliases)

  return absoluteAliases;

}

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
    && (path.length === alias.length || path[alias.length] === '/')
}

const aliases = getAliases();

export const resolve = (specifier, parentModuleURL, defaultResolve) => {
  
  const alias = Object.keys(aliases).find((key) => isAliasInSpecifier(specifier, key));

  const newSpecifier = alias === undefined
    ? specifier
    : path.join(aliases[alias], specifier.substr(alias.length));

  return defaultResolve(newSpecifier, parentModuleURL);
}
Then: node --no-warnings --experimental-modules --es-module-specifier-resolution=node --loader ./custom-loader.mjs index.js
You should have said that in package.json you expected the key aliases and no more _moduleAliases...

I made an npm module using this code that fixes also that problem, check it out here: esm-module-alias

JakobJingleheimer · 2022-11-13T19:10:27Z

Heads up: the node loader API function signatures have changed, and the 2nd argument is an (optional) object (not parentURL). Supplying an argument of invalid type will trigger an exception. Please see the Node.js docs for current signatures.

euberdeveloper · 2022-11-13T20:52:36Z

@JakobJingleheimer

Could you please link me the signature in the doc?
Does this mean that my noduek should behave differently depending on the version of nodejs?

JakobJingleheimer · 2022-11-13T20:59:23Z

https://nodejs.org/api/esm.html#hooks

I'm not sure what version(s) of Node.js the change was backported to, but at the very least, my code from ~2 years ago and what you included in your npm package will not work in the v18 (LTS) or v19 (current) of node.

euberdeveloper · 2022-11-14T01:53:51Z

https://nodejs.org/api/esm.html#hooks

I'm not sure what version(s) of Node.js the change was backported to, but at the very least, my code from ~2 years ago and what you included in your npm package will not work in the v18 (LTS) or v19 (current) of node.

My repo does some tests with also versions of nodejs 18 and 19, and it seems to work

foges · 2023-10-31T07:40:54Z

For anyone else coming across this looking to support wildcard imports replace:

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
    && (path.length === alias.length || path[alias.length] === '/')
}

With

const isAliasInSpecifier = (path, alias) => {
  return path.indexOf(alias) === 0
}

viT-1 mentioned this issue Aug 8, 2019

tsconfig.json/paths not working with ts-node TypeStrong/ts-node#138

Closed

Kehrlann mentioned this issue Oct 7, 2019

Strange?! Can't I alias "pull-fetch" module using "import" but other modules work just fine. #78

Closed

Kehrlann mentioned this issue May 25, 2020

Doesn't work with esm ? #99

Open

siric mentioned this issue Sep 18, 2020

ESM support: soliciting feedback TypeStrong/ts-node#1007

Open

4 tasks

JakobJingleheimer mentioned this issue Jan 6, 2021

json-6 require not working? d3x0r/JSON6#48

Open

cullylarson mentioned this issue Feb 15, 2021

HMR taking much longer with webpack@5 and next@canary vercel/next.js#20760

Closed

richardeschloss mentioned this issue Mar 16, 2021

not able use in node 15 with type: module in package.json #116

Open

kodikos mentioned this issue Feb 10, 2022

when switching to ESM modules, this package no longer works tleunen/babel-plugin-module-resolver#432

Open

lukasz1mroz mentioned this issue Feb 4, 2024

Features/migrate to ES6 syntax idurar/idurar-erp-crm#943

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Not working with native ES modules #59

Not working with native ES modules #59

cullylarson commented May 21, 2019 •

edited

Loading

jdt3969 commented Jun 10, 2019

cullylarson commented Jun 10, 2019

Kehrlann commented Jun 10, 2019

cullylarson commented Jun 10, 2019

kirkouimet commented Nov 27, 2019 •

edited

Loading

eouia commented Sep 1, 2020

JakobJingleheimer commented Jan 5, 2021

JakobJingleheimer commented Jan 5, 2021 •

edited

Loading

TimDaub commented Jan 15, 2021

JakobJingleheimer commented Jan 15, 2021

jaschaio commented Mar 31, 2021

TimDaub commented Mar 31, 2021

szydlovski commented Sep 21, 2021

Kehrlann commented Sep 22, 2021

szydlovski commented Sep 22, 2021

JakobJingleheimer commented Oct 23, 2021

TimDaub commented Oct 23, 2021 •

edited

Loading

JakobJingleheimer commented Oct 23, 2021

TheDirigible commented May 10, 2022

euberdeveloper commented Jul 18, 2022

JakobJingleheimer commented Jul 19, 2022

euberdeveloper commented Nov 13, 2022

JakobJingleheimer commented Nov 13, 2022

euberdeveloper commented Nov 13, 2022 •

edited

Loading

JakobJingleheimer commented Nov 13, 2022

euberdeveloper commented Nov 14, 2022

foges commented Oct 31, 2023

Not working with native ES modules #59

Not working with native ES modules #59

Comments

cullylarson commented May 21, 2019 • edited Loading

jdt3969 commented Jun 10, 2019

cullylarson commented Jun 10, 2019

Kehrlann commented Jun 10, 2019

cullylarson commented Jun 10, 2019

kirkouimet commented Nov 27, 2019 • edited Loading

eouia commented Sep 1, 2020

JakobJingleheimer commented Jan 5, 2021

JakobJingleheimer commented Jan 5, 2021 • edited Loading

TimDaub commented Jan 15, 2021

JakobJingleheimer commented Jan 15, 2021

jaschaio commented Mar 31, 2021

TimDaub commented Mar 31, 2021

szydlovski commented Sep 21, 2021

Kehrlann commented Sep 22, 2021

szydlovski commented Sep 22, 2021

JakobJingleheimer commented Oct 23, 2021

TimDaub commented Oct 23, 2021 • edited Loading

JakobJingleheimer commented Oct 23, 2021

TheDirigible commented May 10, 2022

euberdeveloper commented Jul 18, 2022

JakobJingleheimer commented Jul 19, 2022

euberdeveloper commented Nov 13, 2022

JakobJingleheimer commented Nov 13, 2022

euberdeveloper commented Nov 13, 2022 • edited Loading

JakobJingleheimer commented Nov 13, 2022

euberdeveloper commented Nov 14, 2022

foges commented Oct 31, 2023

cullylarson commented May 21, 2019 •

edited

Loading

kirkouimet commented Nov 27, 2019 •

edited

Loading

JakobJingleheimer commented Jan 5, 2021 •

edited

Loading

TimDaub commented Oct 23, 2021 •

edited

Loading

euberdeveloper commented Nov 13, 2022 •

edited

Loading