Fix Cesium in Node.js 12+.

[kring]

Node.js 12+ requires that ES modules are loaded with import rather than require, and it throws an exception if you get this wrong. An ES module is any module that is named .mjs or whose closest package.json specifies "type": "module" (as Cesium's does).

The Node.js module docs explain a couple of approaches for making a single package that works when imported as either an ES module or as a CommonJS module. But the first requires that the library be natively CommonJS (which Cesium is not), and the second requires two copies of the library (one CommonJS modules, one ES modules) and a dodgy approach to allow the two to share state 🙄. There's no scenario where the library is natively ES modules and can be loaded in CommonJS apps using esm, and short of terrible hackery it appears that this is no longer possible to achieve in Node 12+.

So, this PR does some terrible hackery, as suggested here:
standard-things/esm#855 (comment)
(By the way, the commentor above implemented the check and exception that is causing us problems, as well as large parts of Node's module system in general I believe, so presumably knows what he's talking about! nodejs/node#29492)

This PR also adds a package.json with "type": "commonjs" to the Tools directory so that the .js files there are treated as CommonJS. This is needed for several of our build tools to work in Node 12+. It also renames gulpfile.js to gulpfile.cjs so that file isn't interpreted as an ES module, and it adds an explicit parameter to all the npm run scripts telling gulp to use it. gulp isn't (yet?) able to look for a gulpfile.cjs itself.

Other approaches we could take, not implemented in this PR:

• Remove "type": "module". If we do that, we don't need any of the other changes in this PR and everything will work great in Node 12+. BUT users won't be able to import Cesium as "real" ES modules using --experimental-modules anymore. They'll always have to use esm.
• Ship separate CommonJS and ES builds of Cesium in the npm package. Name the files .mjs or .cjs as appropriate (or put a separate package.json in each version's subdirectory with a different "type"). Ideally break all of Cesium's shared state into separate CommonJS modules as described in Approach 2 in the node.js docs above. That way if one thing imports the CommonJS version and another thing imports the ES version, will "only" have two versions of all the code rather than everything breaking cause there's two versions of the global data.

Fixes #8452

[cesium-concierge]

Thanks for the pull request @kring!

• ✔️ Signed CLA found.
• ❔ CHANGES.md was not updated.
  • If this change updates the public API in any way, please add a bullet point to CHANGES.md.
• ❔ Unit tests were not updated.
  • Make sure you've updated tests to reflect your changes, added tests for any new code, and ran the code coverage tool.

Reviewers, don't forget to make sure that:

• Cesium Viewer works.
• Works in 2D/CV.
• Works (or fails gracefully) in IE11.

[kring]

By the way, be very careful when testing anything with esm. Once it successfully loads a file (e.g. because you changed something), it will cache the version of the file converted to CommonJS. Then future loads of the same file will use the cached version, which Node.js considers to be the right module type. So if you:

1. Make a change such that your test Cesium module loads successfully with esm (e.g. remove "type": "module" from package.json.
2. Run the app once so it gets loaded.
3. Make another change such that now the Cesium module should not load successfully because of the ERR_REQUIRE_ESM error (e.g. re-add "type": "module" to package.json).
4. Run the app again.

The app will run successfully until you rm -rf node_modules/.cache/esm! Ask me how much time that cost me. Actually, please don't. I don't want to talk about it.

[kring]

I made a project to test some different scenarios for importing Cesium in node.js:
https://github.com/kring/cesium-in-node-scenarios

[emackey]

This branch built for me under Node 12 LTS. The npm install phase was pretty noisy with warning messages, but the build seems OK.

I got about 37 warnings from npm install all related to the fsevents module, that all look like this but with assorted filenames on the end:

npm WARN enoent SKIPPING OPTIONAL DEPENDENCY: ENOENT: no such file or directory, rename '.\node_modules\chokidar\node_modules\fsevents\node_modules\yallist' -> '.\node_modules\chokidar\node_modules\fsevents\node_modules.yallist.DELETE'

[kring]

Thanks for taking a look, Ed. I think those warnings happen even in Node 10, don't they?

[emackey]

I may have skipped 10 on my way here, sorry... 😄

[mramato]

Yeah, npm install is noisy and terrible. Those are in 10 as well. I will hopefully have some time to look over this soon.

[mramato]

This is a global change to the node environment, right? Sounds like we don't have a lot of options, just wanted to confirm.

Just to throw out an alternative, we could always use the built version of Cesium in NodeJS and by requiring CesiumUnminified/Cesium.js we would still get extra debugging and such.

[mramato]

Actually, if esm is going the way of the dodo, maybe it's better to just use combined in all cases now and we can revisit going full import later on (Node 14?)

[kring]

Great idea! I got rid of the hack and made index.cjs always load the built version, either Build/Cesium/Cesium.js or Build/CesiumUnminified/Cesium.js depending on the NODE_ENV.

I also hooked up sourcemaps so it still looks like separate source file while debugging. I couldn't get it to work in the minified build for some reason, though. Uglify/Rollup are supposed to be able to do sourcemapping in this scenario, but the generated sourcemaps were rubbish. Step into a Cesium function in a release build and you'd end up somewhere totally random instead of in the function you're actually in.

If you use Node 12 and proper ES module support, you'll use the ES modules directly, with no transpilation by esm or other shenanigans. However, the debug pragmas won't be removed in this scenario. That's unfortunate, but a problem for another day, I think.

[mramato]

Awesome, thanks again for your work on this. We talked offline, but just to summarize for anyone else, rollup-plugin-strip-pragma can be used for pragma removal, debug sourcemaps are still enabled, release will have to wait for a different PR to figure out the problem.

[mramato]

We can revert all of the -f gulpfile.cjs and just tell gulp to use it by default, see https://github.com/CesiumGS/cesium/pull/8541/files#diff-a663dcab4db6e595455e0c837b86ee03

[kring]

Nice, done.

[mramato]

Thanks for the exemplary PR description and details @kring.

Pretty sure I'm okay with all of this except for the those couple of questions/comments.

We could potentially do a node-only release of Cesium with this fix once it's in master since it's affecting a lot of people at this point

[tmcintire]

I'd love to see this merged in, we could really use this change! :)

[mramato]

@tmcintire this will definitely go out with the next release on April 1. @kring do you have time to help finish this up? If not I can take it over.

[kring]

@kring do you have time to help finish this up? If not I can take it over.

Yep, thanks for the reminder! I'll do it today.

[mramato]

Everything looks great, just want to do some final testing once CI passes and than I'll merge.

[mramato]

Did a quick test with node 12 and 13 and was able to import Cesium in both cases as both commonjs or ES modules. (Node 12 still required a flag, 13 did not).

Also tested a rollup based app using node-resolve with Cesium ES6 and that worked as well.

Going to merge so this has some time to stew in master.

Thanks again @kring!

[shunter]

@kring I noticed that after this PR the copyright banner is no longer included in the built minified files (for example, check today's release zip) despite using the banner option. Maybe that option doesn't work correctly with minified bundles?