Can't use mathjs without package bundler

[GreenImp]

The problem

I'm trying to use mathjs inside node, using ES modules, but without a package bundler (e.g. Webpack, Rollup).

When I do:

import { evaluate } from 'mathjs';

console.log(evaluate('1+2'));

It throws:

import { evaluate } from 'mathjs';
         ^^^^^^^^
SyntaxError: The requested module 'mathjs' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.
For example:
import pkg from 'mathjs';
const { evaluate } = pkg;

From what I can see, the module entry in package.json is only used for package bundlers. Without bundlers, node will look at the main entry, so it tries to load the CommonJS / UMD script.

Node does support mixing ES and CommonJS modules, but expects a single, default export.
Changing the import to this works, because it correctly loads the CommonJS module:

import mathjs from 'mathjs';

console.log(mathjs.evaluate('1+2'));

So that's a kind of workaround, but it does mean that it's not using ES modules in an ES module project.

But the bigger issue is that it's now incompatible if I choose to use to use a bundler because the bundler will use the ES module version, which has no default import.
In particular, this causes my Jest tests to fail. In my test I have something like:

TypeError: Cannot read property 'evaluate' of undefined

I think that conditional exports may be a solution. It seems to allow you to tell node which files to use for CommonJS require and which to use for ES import.

Potential solution

I've not had a proper play with it, this seemed to work for me:

Change the package.json to this, to tell node to use the esm files for import statements, and es5 files for require:

{
  "main": "./main/es5",
  "module": "./main/esm",
  "exports": {
    "import": "./main/esm/index.js",
    "require": "./main/es5/index.js"
  },
  // this is required, otherwise the `exports` doesn't work
  "type": "module"
}

But because the type is now module, node will now complain when trying to use CommonJS.
To sort that, you can either rename all of the CommonJS files to have the .cjs extension, or to add a package.json in the root directories containing the CommonJS files (e.g. main/es5/, lib/), with just the following:

{
  "type": "commonjs"
}

I'm not sure if doing this has any adverse affects on other parts of the library though, but I'm happy to get a PR in for those changes.

I thought it seemed similar to #1766. Not quite the same issue, but possibly the same resolution, perhaps used with Subpath Exports.

[josdejong]

Thanks Lee! It would be great to get mathjs working directly as ES module without need for bundlers! There are a lot of ways the library can be used, so we should thoroughly test all these options. Help getting this sorted out would be very welcome!

I wasn't aware of the conditional exports field that is available in package.json (and I'm not sure if this is node.js only, or also supported by bundlers like Webpack/Rollup/etc).

I can think of:

• Use in new node.js environments
• Use in old node.js environment (commonjs only)
• Use directly as ES module in a browser
• Use via Webpack, with either ES module or commonjs config
• Use via Rollup
• ...

I think we should think through a "real" solution and not go for workarounds, I'm afraid workarounds will come back to bite us. It makes sense to me to give all files the correct, explicit extension: *.mjs for the ES module builds, and *.cjs for the commonjs builds. All imports should explicitly note the file extension too, else you can't load it as plain ES module. I hope there are tools to do these things automatically for us: renaming files and updating imports etc. And we should set up sensible defaults in package.json, such that the library works out of the box in most environments.

Your help would be very welcome!

[GreenImp]

Thanks for your response! I only came across conditional exports recently myself, as I'm planning on implementing it in my own library, to solve a similar issue.

I'm definitely interested in getting this sorted. Are you able to clarify for me if my understanding of the uses of each file?

• /dist/math.js / /dist/math.min.js: Used in browsers via the script tag.
• /es/*: ES module import for use in browsers that support it, and also newer NodeJS
• /lib/*: CommonJS module require for use in NodeJS
• /main/es5/index.js: CommonJS main entry point
• /main/es5/number.js: CommonJS entry point with less functionality
• /main/esm/index.js: ES module main entry point
• /main/esm/number.js: ES module entry point with less functionality
• /src: Source files, using CommonJS, that are compiled, with Gulp, to the others

I'm not entirely certain what the difference with the number.js files is?

Are there Any relevant files / uses that I've missed?

I think people using Rollup / Webpack / etc. isn't too important, as I think they'll grab whatever node is told to. So, if we add the exports to tell it what to use, this should just work 🤞

My thoughts on adding package.json files specifying the type, rather than renaming files to .cjs was partly because I wasn't sure if you'd be happy to rename them all, but also because I hadn't realised before that there's a separate, compiled, browser build; browsers don't understand the .cjs extension. If you'd rather rename, then that's probably better.

That being said, I've not much experience with using Grunt, and I'm not sure how to change the file extension that it generates.
The same goes for adding the file extension to the import statements in the ESM modules.

[GreenImp]

Thought it worth mentioning that I've just implemented exports on my RPG Dice Roller: https://github.com/GreenImp/rpg-dice-roller/pull/174/files#diff-b9cfc7f2cdf78a7f4b91a753d10865a2

"exports": {
    ".": {
      "import": "./lib/esm/bundle.js",
      "require": "./lib/umd/bundle.js"
    },
    "./lib/umd/bundle.js": {
      "require": "./lib/umd/bundle.js"
    }
  },

And it seems to be working nicely.

The "." entry is the root, so it handles importing the base package like:

import('package-name');

The next bit;

    "./lib/umd/bundle.js": {
      "require": "./lib/umd/bundle.js"
    }

Allows CommonJS access to the UMD bundle, but will actually throw an error that the import is invalid, if you try and use ESM import, which is handy:

// this works
const value = require('package-name//lib/umd/bundle.js');

// this doesn't
import value from 'package-name//lib/umd/bundle.js';

So it should definitely be possible to set up mathJS to export both the main/{esm|es5}/index.js and main/{esm|es5}/number.js.
Something like:

{
  "main": "./main/es5",
  "module": "./main/esm",
  "exports": {
    ".": {
      "import": "./main/esm/index.js",
      "require": "./main/es5/index.js"
    },
    "./number": {
      "import": "./main/esm/number.js",
      "require": "./main/es5/number.js"
    }
  },
  "type": "module"
}

This should allow you to import both:

// using ESM
import { sqrt } from 'mathjs';
import { sqrt } from 'mathjs/number';

// using CommonJS
const { sqrt } = require('mathjs');
const { sqrt } = require('mathjs/number');

And that should work with package bundlers and without.

[josdejong]

Your explanation of the folders dist, es, lib, and src is correct. The idea of the main folder was indeed to offer one place with all the relevant index files. I'm not sure if it's actually helpful though.

I'm not entirely certain what the difference with the number.js files is?

The number.js files contain the mathjs functions but with number support only, making it lightweight. No matrix, unit, bignumber, fraction, complex, etc support.

About renaming files: I assume in the future everything will become ES modules with the *.mjs extension. For the time being we have to support a hybrid solution. We can see the commonjs exports under /lib/ as the exception here, which may or may not be renamed to *.cjs. I'm really not sure what is the best approach, but I think it's good to have a clear dot on the orizon.

So it should definitely be possible to set up mathJS to export both the main/{esm|es5}/index.js and main/{esm|es5}/number.js.

That sounds promising! So this will work for node.js for sure. Will it also work with Webpack/Rollup?

[GreenImp]

In terms of the main folder, it might be beneficial to move main/esm/*.js into the /es/ directory, and the main/es5/*.js into the /lib/ directory.
This means that it's all in one place, and the index.js files then become the default loaded for both of those directories.

I'm not sure what's best with filenames either.
I'm not sure about .mjs for ESM as I think that extension is going to become legacy fairly quickly as CommonJS is dropped in favour of ESM, meaning that .js is just assumed to be ESM. Whereas .cjs will probably stick around a lot longer. Obviously I'm just theorising though.
I'm guessing the files are being named the same as the source files? I can't see anything in the Gruntfile that specifies naming conventions. It's slightly different to how I handled it, where all of the files where bundled into a single file that exports everything. Not sure that's better or worse though, but I think that way may make it more difficult for tree shaking.

I think bundlers will still work exactly as they currently do; look at the package.json main and module entries for CommonJS and ESM respectively.
I don't think anything will change, provided that those two properties are left as is, in the file. But it'll obviously need some testing to make sure.

I'll start working on a PR for the package.json changes and see how it goes.

Out of interest, is the number.js functionality comparable with the functionality in https://github.com/josdejong/mathjs-expression-parser ?
I was using that initially, as it's smaller, but using the main library now for the ESM support. But numbers.js didn't seem to provide an evaluate method, which I need, so I can do things like:

const result = evaluate('4+1'); // result === 5

[josdejong]

In terms of the main folder, it might be beneficial to move main/esm/.js into the /es/ directory, and the main/es5/.js into the /lib/ directory.
This means that it's all in one place, and the index.js files then become the default loaded for both of those directories.

Makes sense, fine with me 👍 (though a breaking change so we need to publish as a new major version)

About the file naming: maybe we should search a bit on internet for best practices in this regard. We could say that .js is ES modules, and .cjs is CommonJS, and define "type": "module" in package.json.

The build script of mathjs outputs a number of different things: a bundle, a minified bundle, a directory with ES modules code, and a directory with commonjs code.

I'll start working on a PR for the package.json changes and see how it goes.

Thanks!

Out of interest, is the number.js functionality comparable with the functionality in https://github.com/josdejong/mathjs-expression-parser ?

The mathjs-expression-parser is indeed redundant since [email protected], which introduced the lightweight mathjs/number functions and tree shaking with ES modules. The mathjs/number actually does contain the evaluate function.

[GreenImp]

Ah, not sure how I missed the evaluate method in the numbers file. Must have misspelt it or something. Serves me right for working on it so late.

I've started having a play around with this, but have hit an initial issue;

For the exports property to work, it seems as though you also have to have "type": "module". But this causes issues running the gruntfile, because node now expects it to be an esm module.

I'm still looking into it, and trying to find the best way of handling it all, but thought I'd give an update on progress.

[GreenImp]

I've created PR #1941 with some changes to fix this. The description on it is pretty lengthy, but it hopefully describes everything that I've done.

Feel free to pick it apart and let me know your thoughts.

[josdejong]

Thanks for your your PR Lee! I'll look into it asap but I have little time these weeks unfortunately.

Just for reference: I recently came across this article which is about this exact issue: https://redfin.engineering/node-modules-at-war-why-commonjs-and-es-modules-cant-get-along-9617135eeca1

[GreenImp]

No worries, I know it can be difficult to find time for stuff. Just take a look when you get the chance 😄

Thanks for the link! It's a really interesting read.
It's good to see that they're suggesting the use of a package.json in the ESM directory, to specify the type. It also looks like I've done the right thing with the exports property in the root package.json.

It does sound like if the source was written in CommonJS, rather than ESM, you may be able to get the generated ESM code to be smaller, by "wrapping" the CommonJS files, rather than duplicating the code. Which is really interesting!
It doesn't look like you can compile the CommonJS as a thin wrapper for the ESM version though, unfortunately.
I'm not suggesting to re-write the package in CommonJS though, I like ESM 😉

[GreenImp]

Also, feel free to be ruthless with your review. I'm happy to change things, or scrap it, if it's not what you're after.

[josdejong]

Thanks for your patience, appreciated!

Just for reference, more background information about cjs/esm I came across recently: https://nodejs.org/dist/latest-v14.x/docs/api/esm.html#esm_dual_commonjs_es_module_packages

[cshaa]

Was this fixed by #1941, or is it still an issue?

[brcolow]

I'd also love to know if anyone has figured out how to use math-js in the browser without a package bundler. Trying to do that myself and running into the same issues as OP.

Note: Using npm install mathjs and then using <script src='./node_modules/mathjs/lib/browser/math.js'></script> does work but it would be really nice to be able to use import { inv } from 'math-js' in the browser, without a bundler.