"gatsby build" breaks when dependencies use es6 code (which Uglify doesn't support)

[szimek]

Description

After upgrading Gatsby to 1.9.167, I'm getting the following error when I try to run gatsby build:

error Generating JavaScript bundles failed

  Error: component---src-templates-post-jsx-dbdaebf346a9292b96a8.js from UglifyJs
  SyntaxError: Unexpected token: name (isLastCharLower) [./~/camelcase/index.js:4,0]

The line mentioned in the error contains keyword let. Looks like (at least) dependencies are no longer compiled to ES5 and UglifyJS version used by Gatsby doesn't understand ES6+, so it throws an error.

The same app works fine with 1.9.166. It's most likely caused by some changes introduced in #3369, other changes introduced in this version don't look suspicious.

To replicate this issue create a new Gatsby project, run yarn add camelcase, import and call camelcase in any component and run gatsby build.

[617dev]

i am also seeing this. except for me it's running out of memory when trying to build the JS bundles, however pinning to the version you mentioned 1.9.166 works fine...
@szimek when you say some code isn't being compiled to ES5, do you know which code? I'd like to see if ES5-ing it would fix the error with the newer versions to prove your theory...

[szimek]

In my case it's caused by camelcase package (which is a dependency of a markdown to html library I'm using), but there are others available only in ES6 as well. This error has already been reported sindresorhus/camelcase#21 and it won't be "fixed" - more info about that sindresorhus/ama#446, which makes perfect sense from the maintainer point of view.

However, this means that all dependencies should also be compiled by Babel to ES5, because they might be written in ES6, so even if Gatsby started using UglifyJS2, which does support ES6 syntax, it would break in older browsers that don't support ES6.

[ryami333]

Experiencing this too - big dealbreaker as far as I'm concerned.

[KyleAMathews]

Gatsby has never compiled code in node_modules to es5. If you're seeing problems then it's because either an existing dependency of yours started shipping es6 code or (more likely) you added a new dependency with es6 cod which broke uglify.

CRA will be compiling code in node_modules w/ babel starting in their v2 which we can consider as well.

facebook/create-react-app#3776

@szimek that PR only affects the Typescript plugin. We haven't changed anything related to Babel in a long time.

@alexjsdev are you using the typescript plugin?

[ryami333]

I would have thought so too @KyleAMathews, but I can verify that it builds correctly on Gatsby 1.9.166 but not on the latest (1.9.178 at time of writing), so whether or not the dependency in question has changed is irrelevant (and it hasn't, I checked).

[KyleAMathews]

@ryami333 fair enough — could you post a link to a site reproducing this problem? All the example sites and gatsbyjs.org are still building correctly AFAIK so something reproducing this would be great.

[ryami333]

I'll see what I can do, but it might take a little while. In the meantime, I verified that the break is between builds 1.9.166 and 1.9.167.

[617dev]

@KyleAMathews no we are not using typescript, just the out of the box gatsby starter setup, with what was the latest Gatsby at the time which was 1.9.176 (i believe). We would get an UglifyJS running out of memory error when Gatsby would try to compile our JS bundles, but then i found this thread and based on @szimek suggestion tried pinning the version to 1.9.166 and voila! it works... I'm not sure what changed, do you have a changelog for each version, or like a tagged merged commit for each where I could try to help figure this out? I really don't want to leave the version pinned.

On a side note, I tried updating to the latest a few days ago (1.9.177) and although gatsby would successfully run develop if i ran a build it would hang on createPagesStatefully forever... so different behavior (than the uglify JS error but still no functional build)

Thanks for all your help! And btw Gatsby is truly genius!

[617dev]

@KyleAMathews just saw that there's a newer version 1.9.178 and it looked like there was a fix merged for the hanging i was talking about... well it's true it doesn't hang there anymore but this is back now 😢:

error Generating JavaScript bundles failed

  Error: app-45a58aeb1a29bc012abd.js from UglifyJs
  RangeError: Maximum call stack size exceeded```

[ryami333]

@alexjsdev is your issue actually related to the OP? Have you verified that your dependency is indeed ES6?

[ryami333]

@KyleAMathews I've created a demo repo here, based on gatsby-starter-default.

• Commit 1, initial commit (fresh gatsby-starter-default).
• Commit 2, demonstrate successful build with ES6 dependency on 1.9.166.
• Commit 3, demonstrate failing build with 1.9.167.
• Commit 4, demonstrate failing build on latest release.

[szimek]

@KyleAMathews

Gatsby has never compiled code in node_modules to es5.

You're right. Turns out that Gatsby 1.9.166 and earlier versions silently skip the minification of a file, if it uses ES6 syntax after bundling, e.g. when I use camelcase library in one of pages. If I use it in pages/index.js, all files will be minified except public/component---src-pages-index-js-xxx.js. Gatsby 1.9.167 and later versions throw an error in this case.

According to facebook/create-react-app#1125 (comment) it's not safe to run Babel 6 on node_modules and Babel 7 will be added in Gatsby v2, right?

Here's PR that adds compiling dependencies with babel-preset-env to create-react-app: facebook/create-react-app#3776

[danoc]

I can also confirm that this is happening.

In my particular case, it is caused by a let used in buble, a dependency included within react-live.

Downgrading to 1.9.166 does fix the problem. Haven't dug into what the proper fix should be.

[KyleAMathews]

@szimek yeah, planning on copying CRA's implementation for Gatsby v2.

[KyleAMathews]

Folks running into this — what do think we should do? Remove the error? Add easy way to compile problematic packages? Just removing the error doesn't really solve much as now you're shipping unminified es6 code which is a) slower cause it's unminified and b) will break older browsers that don't support whatever es6 features are used in the code.

[ryami333]

Why isn't using an es6-compatible version of Uglify an option? Get Outlook for Android<https://aka.ms/ghei36>

…

________________________________ From: Kyle Mathews <[email protected]> Sent: Tuesday, February 6, 2018 4:14:17 PM To: gatsbyjs/gatsby Cc: Mitch Ryan; Mention Subject: Re: [gatsbyjs/gatsby] "gatsby build" breaks when dependencies use es6 code (which Uglify doesn't support) (#3780) Folks running into this — what do think we should do? Remove the error? Add easy way to compile problematic packages? Just removing the error doesn't really solve much as now you're shipping unminified es6 code which is a) slower cause it's unminified and b) will break older browsers that don't support whatever es6 features are used in the code. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#3780 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AL5z_PbtaWjW47jYiSPvh-WYBO8U7MFfks5tR8OJgaJpZM4Ry0kI>.

[KyleAMathews]

That could be a possibility too — I haven't researched it. Actually, just researched it :-) and it looks like it might require webpack v3 — so we should do this upgrade along with the remainder of the Gatsby v2 work facebook/create-react-app#3618

[szimek]

Why isn't using an es6-compatible version of Uglify an option?

There are 2 issues. When Gatsby switches to babel-preset-env it will need to use ES6-compatible minifier (most likely UglifyJS2), because user's code can end up with ES6+ syntax as well. However, if I configure babel-preset-env to support IE9 or Safari 9, I'd expect all code to work in these browsers, including third-party code that might use ES6+ syntax, not just the code I wrote.

[Kikobeats]

@KyleAMathews Could be possible setup an Uglify ES6-compatible on gatsby-node.js?

Just as workaround to avoid not update gatsby dependency.

Looks like, if I don't update gatsby this error appears:

GraphQL Error There was an error while compiling your site's GraphQL queries.
  Error: Cannot use GraphQLSchema "[object Object]" from another module or realm.

Ensure that there is only one instance of "graphql" in the node_modules
directory. If different versions of "graphql" are the dependencies of other
relied on modules, use "resolutions" to ensure only one version is installed.

https://yarnpkg.com/en/docs/selective-version-resolutions

Duplicate "graphql" modules cannot be used at the same time since different
versions may have different capabilities and behavior. The data from one
version used in the function from another could produce confusing and
spurious results.

EDIT: In my case looks a very isolate problem related with a subdependency from react-live
I opened an issue FormidableLabs/react-live#61. If you want to reproduce the problem just:

git clone -b update [email protected]:microlinkhq/www.git
npm install && npm run build

[KyleAMathews]

@Kikobeats it's best to fix your dependency by compiling it to es5. Not updating Gatsby just means your build succeeds not that the underlying problem, that your code isn't minifying and that it's not runnable on older browsers, is fixed.

[KyleAMathews]

Working on fixes for this in Gatsby v2. We'll be compiling all code in node_modules + upgrading to the latest version of uglify.

[chadwatson]

@m-allanson thank you so much for your help!

[RobertoMSousa]

Did anyone found a solution for this? I'm having the same problem. Here are my gatsby.config

module.exports =  {
  siteMetadata: {
    title: '4Humans',
  },
  plugins: [
    `gatsby-plugin-react-next`,
    `gatsby-plugin-core-js`,
    'gatsby-plugin-react-helmet',
    `gatsby-plugin-sass`,
    `gatsby-plugin-typescript`
  ],
}

and my dependencies

"dependencies": {
    "gatsby": "^1.9.166",
    "gatsby-link": "^1.6.40",
    "gatsby-plugin-catch-links": "^1.0.22",
    "gatsby-plugin-core-js": "^1.0.2",
    "gatsby-plugin-google-analytics": "^1.0.31",
    "gatsby-plugin-mailchimp": "^2.2.3",
    "gatsby-plugin-react-helmet": "^2.0.11",
    "gatsby-plugin-react-next": "^1.0.11",
    "gatsby-plugin-sass": "^1.0.26",
    "gatsby-plugin-sharp": "^1.6.46",
    "gatsby-plugin-typescript": "^1.4.20",
    "gatsby-source-filesystem": "^1.5.36",
    "gatsby-transformer-remark": "^1.7.41",
    "gatsby-transformer-sharp": "^1.6.26",
    "react-helmet": "^5.2.0",
    "react-particles-js": "^2.2.0",
    "tslint": "^5.10.0",
    "typescript": "^2.9.2",
    "typography-theme-grand-view": "^0.15.10"
  },

[natesjacobs]

Hi all. I'm having similar issue with 'query-string' package. I'm fine with not having my site working on older browsers... what are people doing to get to solution where it just skips over the ES6 packages? I saw @KyleAMathews is pushing out v2 (woohoo! thanks dude!) but that's causing other errors for me so wanted to find a solution while v2 settles in :)

[RobertoMSousa]

@natesjacobs I solved my problem my removing async/await from my app. Apparently, the v1 does not support it yet. Check if you have any and change it to the good old callbacks ;-)

[fsegouin]

@natesjacobs I ran into the same issue and had to use query-string@5.

[greypants]

I had same issue upgrading to to a module dependency that uses let and const. The solution that finally worked for me was whitelisting specific modules in the babel loader via the weback config. I used the example in the docs as a starting point, and ended up with something like this in gatsby-node.js:

exports.modifyWebpackConfig = ({ config, stage }) => {
  // https://www.gatsbyjs.org/docs/add-custom-webpack-config/#modifying-the-babel-loader
  const program = {
    directory: __dirname,
    browserslist: ["> 1%", "last 2 versions", "IE >= 9"]
  };

  return generateBabelConfig(program, stage).then(babelConfig => {
    config.removeLoader("js").loader("js", {
      test: /\.jsx?$/,
      exclude: modulePath => {
        return (
          /node_modules/.test(modulePath) &&
          // whitelist specific es6 modules
          !/node_modules\/(react-docgen|ast-types|recast)/.test(modulePath)
        );
      },
      loader: "babel",
      query: babelConfig
    });
  });
}

[greypants]

We'll be compiling all code in node_modules

@KyleAMathews Won't that significantly increase the build time? Sticking with exclude and just whitelisting (like I've done above) prevents babel from evaluating more than it needs to, right? Maybe just needs to be documented. Plus I think the webpack + uglify upgrade takes care of this specific issue anyway.

[KyleAMathews]

Yes — we've actually decided to wait and see a bit more on the issue of compiling node_modules. Both because of the slowdowns and because there's a lot of active discussions around how it should work.

[robwalkerco]

If anyone else comes across this issue when they are using Gatsby V2, the above examples will not work due to Gatsby API changes, but the below example did (at least for me).

exports.onCreateWebpackConfig = ({ loaders, actions }) => {
  actions.setWebpackConfig({
    module: {
      rules: [
        {
          test: /\.js$/,
          exclude: modulePath =>
            /node_modules/.test(modulePath) &&
            // whitelist specific es6 module
            !/node_modules\/@papertrailio\/js-core/.test(modulePath),
          use: loaders.js(),
        },
      ],
    },
  })
}

In the above example, the @papertrailio/js-core module included Flow definitions, so adding the above code to gatsby-node.js, and adding the gatsby-plugin-flow plugin to gatsby-config.js worked perfectly.

[angeloashmore]

Could this functionality be outsourced to a plugin? Something where you could just provide a list of node_module packages that need to be transpiled. In the implementation, it just does the onCreateWebpackConfig above.

[KyleAMathews]

Yeah! This should totally be a plugin. Anyone up for creating it?

[robwalkerco]

@KyleAMathews I will have a crack at a plugin for Gatsby V2 using something like my example above (#3780 (comment))

Perhaps I should call it gatsby-plugin-es6-webpack-package?

[KyleAMathews]

@robwalkerco awesome! How about maybe gatsby-plugin-compile-es6-packages?

[robwalkerco]

@KyleAMathews Ah, thats a good name.

I published it as gatsby-plugin-es6-webpack-modules a few hours ago and the Gatsby Plugins directory has picked it up already at https://www.gatsbyjs.org/packages/gatsby-plugin-es6-webpack-modules/

Could try to get the name changed though.

If anyone else finds this issue, then the plugin will help you to customise the webpack config to compile es6 modules in node_modules.

Open to PR's to improve the readme etc

[robwalkerco]

After sleeping on it, I've decided that @KyleAMathews plugin name makes much more sense, so have moved the plugin to https://www.npmjs.com/package/gatsby-plugin-compile-es6-packages

I hope you find it useful 😄

Also closing this issue now, as using this plugin and configuring it with the npm packages that have es6 code in them will allow them to be compiled by Babel.

[dustinhorton]

I'm working in a 'monorepo' and trying to figure out if I need @robwalkerco's plugin. The app is structured like:

monorepo-app/
  |- packages/
     | - common/
     | - gatsby-app-1/
     | - gatsby-app-2/

…where common includes shared components, libs, etc.

Should I be adding common as a package to this plugin if I'm using es6, or is there something more to it? gatsby build doesn't error with the UglifyJS message mentioned here.