Add .mjs support back to webpack (it has native ESM support)

[Timer]

Fixes #5234.
Closes #5257.
Closes #5235.

Webpack was never the problem with .mjs support, so we probably shouldn't of removed it.

We still want to figure out a story for Jest, we're just not sure what that is yet.

---

I opted to resolve .mjs before .web.js this time, does this seem OK?

[gaearon]

Can we make sure we test this with packages that created problems before? And that they work in Jest due to the fallback?

[Timer]

We can use the existing behavior tests and add a few more, making sure GraphQL & friends work.

[Timer]

Here's my understanding:

The ecosystem is currently embracing experimental mjs support due to widespread adoption of ESM, fueled by the early days of Babel.
Turning off this support holistically is not an option (like we tried to do in v2). It's just not practical, and still begs questions about how ESM in .js files should be handled.

.mjs is the specification and nitty-gritty details on how ESM will work in Node, meaning we should align as closely to this behavior for .mjs files.
This specification currently mandates that ESM can only import the default export from CommonJS.

This constraint has been applied to the following systems:

1. TypeScript 2.7 added a new mode close in line to this proposal.
2. webpack 4 added more spec-compliant handling of ESM.
3. Babel added a spec mode, in line with above behavior, only permitting importing the default export from CommonJS.

webpack 4's default behavior is to enforce spec for files with the .mjs extension, only allowing the default import to be imported from CommonJS.
webpack 4 still allows .js files containing ESM to import "named exports" from CommonJS.

This behavior aligns closely with what we currently see in the ecosystem:

1. Legacy ESM interop/previously compiled ESM modules are published under .js extensions
2. Node-specified interop between ESM and CJS will lives under .mjs extensions, which has strict behavior

I believe setting javascript/auto will just make the .mjs adoption story worse, either forcing Node to support bad semantics or dramatically increasing the churn when .mjs is finalized.

Given all the information provided above, I suggest we take the following path:

1. We add support for .mjs files back to webpack, and resolve them before all other extensions
2. We leave .mjs support on in strict mode -- this will ensure proper compatibility when Node finalizes the specification and we have to start wondering about how usage with Jest will look
3. We attempt to add no support for mjs to Jest or our Jest configuration until the Node specification is no longer experimental and ships in a LTS version of Node
   • This will encourage users to continue shipping CommonJS versions of their package while the specification settles. If the fickle finger of fate ruins everything, we can always revert back to CommonJS. 😄
4. We discourage users from using the .mjs extension in their project code, as it will be problematic to test

Potential issues:

1. The browser field is recognized before module, meaning if it points to index.js and index.js imports ./foo, webpack will import ./foo.mjs before ./foo.js.
   • This behavior could be problematic if the published package differentiates behavior between mjs and js files, but this is not likely
2. Users start shipping packages without CommonJS available under the main key in package.json as a result of this behavior, further reducing the ability to test with Jest (we're already seeing this, probably not a concern)
3. I'm OK with this causing some churn for a few packages which may have opted into using mjs but didn't vet their dependencies having properly exported/available .mjs distributions. These dependencies might have to replace named imports with default imports.

[jaydenseric]

The browser field is recognized before module, meaning if it points to index.js and index.js imports ./foo, webpack will import ./foo.mjs before ./foo.js.

I'm not following the issue… .mjs is suitable for bundling via Webpack for use in browsers too. Can't a dependency of a dependency also provide a browser entry, if necessary?

[jaydenseric]

Do you mean the browser entry will start importing from the server files by accident? That shouldn't happen with this structure:

lib/
├── server/
│   ├── index.mjs
│   └── index.js
└── browser/
    ├── index.mjs
    └── index.js

{
  "browser": "lib/browser",
  "main": "lib/server"
}

[Timer]

The browser entry may not be pre-bundled into a separate tree or dist file.
If it's a .js file that requires other files that don't specify an extension, it'll be automatically promoted to .mjs.

I'm imagining a package.json like this:

{
  "main": "index",
  "module": "index.mjs",
  "browser": "index.js"
}

And a result of this:

Just a thought.

edit: we posted at the same time

Yes, that structure fixes it. But graphql is a perfect example of these files being shipped side by side.

https://unpkg.com/[email protected]/language/

---

I mentioned (maybe not clearly) that I don't think this would happen often, so it's probably nothing we need to be worried about. It's just a possible (not probable) concern.

[jaydenseric]

Not following what you mean with graphql, they are meant to be sibling .mjs and .js files. It does not have a browser field, so both .mjs and .js files should be ok to bundle for use in a browser. .mjs is preferable, so Webpack can treeshake.

[Timer]

I used graphql just as an example, pretend it had a browser field and pretend that behavior was different because .mjs was intended for Node and .js was more general purpose.
This is a completely fabricated hypothetical, and one I don't think we need to be concerned about.

It's just a possible (not probable) concern.

The reason I say this is a concern is because (AFAIK) Node will resolve .mjs before .js or .js before .mjs depending on the calling file type -- this behavior isn't possible in webpack, you get one resolution order.

[jaydenseric]

Just a side note, but I think with an extensionless main entry and sibling .mjs/.js files, the module entry is kind of redundant, since webpack would have got to the same .mjs file following either entrypoint. I might remove the module field from my packages at some point.

[jaydenseric]

Also, this example was incorrect:

{
  "main": "index.js",
  "module": "index.mjs",
  "browser": "index.js"
}

Node.js only looks at the main field, so it needs to be extensionless:

{
  "main": "index",
  "module": "index.mjs"
}

^ in that example you might have even gotten away with no main or module entry at all; Node.js and Webpack look for index anyway I think.

[Timer]

I'm definitely a proponent of dropping module once main is extensionless with side-by-side files.

[Timer]

I'm going to land this because it aligns with my latest understanding of this.
If we uncover more information or this breaks everything, I'm more than happy to make .mjs use javascript/auto and instead, make this a breaking change in the next version of CRA.

For now, we'll classify this as a bug fix. Cases where this would be considered breaking probably aren't working right now anyway.

We still need to come up with a good testing story for this (e2e).

[Timer]

Tested via #5263.

[jdalton]

@Timer

TypeScript 2.7 added a new mode close in line to this proposal.

That mode aligns TS more with traditional Babel in terms of the __esModule property, enabling support for named exports of CJS modules, and more correct handling of namespace objects. So not related to .mjs or --experimental-modules rules.

Given all the information provided above, I suggest we take the following path:

1. We add support for .mjs files back to webpack, and resolve them before all other extensions
2. We leave .mjs support on in strict mode -- this will ensure proper compatibility when Node finalizes the specification and we have to start wondering about how usage with Jest will look

It would be wonderful if there was a way to give a heads up to folks that using .mjs, in the context of Node, is experimental and should not be used in production (including bundles). I suspect some may have read to use .mjs for ESM code and may not be aware of its complications in Node or tooling.

3. We attempt to add no support for mjs to Jest or our Jest configuration until the Node specification is no longer experimental and ships in a LTS version of Node
   • This will encourage users to continue shipping CommonJS versions of their package while the specification settles. If the fickle finger of fate ruins everything, we can always revert back to CommonJS. 😄

👏

4. We discourage users from using the .mjs extension in their project code, as it will be problematic to test

I've done something similar, but less explicit, with the esm loader by limiting functionality. There is no way to enable CJS interop (module vars, mocking, stubing, spys, code coverage, APM, etc.) or experimental features (like top-level await) with .mjs. More explicit opt-ins, like Node's experimental flag, may be needed for tooling like esm, Babel, and webpack to help inform developers when they're stepping into experimental territory for their packages and applications.