Support reading large stats.json files

[henry-alakazhang]

Node has a string size limit of 500MB and the current JSON.parse(fs.readFileSync(...)) logic to read stats.json files fails for files >500MB.

This PR changes the approach for stats parsing to use a stream, similar to this PR for webpack-cli in creating the file: webpack/webpack-cli#2190

I've tested it against a local repo with a 700MB stats.json and it works when it didn't before. I would add a test, but I don't know about adding that large of a file 😂 Could look into generating one if you would like.

[cblnco]

Is there a possibility this feature gets merged @th0r, @valscion?
I know that the workaround would be to use the plugin instead of the CLI. However, I'm on a scenario where my project doesn't manages all the Webpack settings directly and I can't use this plugin.
It would be really really nice to use the CLI too in this case.

[valscion]

Yeah this looks like something that could work! I seem to have forgotten that this PR exists, sorry about that 😳

@henry-alakazhang would you be open to:

• recreating this pull request on top of the current master branch
• use the latest version of @discoveryjs/json-ext that exists
• create a changelog entry as "Improvement" category
• signing the CLA

We can make do without a test for this as long as the existing tests pass.

[gavinbarron]

@valscion this seems to be stalled, and it's something that we could make use of for tracking work to produce smaller app bundles when using kiota to build TypeScript SDKs for HTTP APIs.

If @henry-alakazhang is not able to progress this contribution I'd be happy to pick it up and move it forward.

[valscion]

Yeah feel free to pick this change up move it forward @gavinbarron ☺️

[cblnco]

Now that I have bandwidth I could also help moving forward with this contribution @valscion, @henry-alakazhang, @gavinbarron.
I will keep an eye on this to know who ends up integrating these changes.

[henry-alakazhang]

Hey whoops sorry, completely missed this. I'll update the PR.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: henry-alakazhang / name: Henry Zhang (e995d16)

[henry-alakazhang]

Updated with latest version of json-ext. I've refactored the code to use async/await because I wasn't sure how readStatsFromFile would actually throw errors.

Remains untested as I can't run yarn test for some reason - the npm i to install the test webpack versions doesn't want to run regardless of which Node version I'm using (erroring with Maximum call stack size exceeded).

edit: tests pass locally. also can confirm it still works with a large file

[valscion]

Thanks, this looks great and the tests still pass in CI so I'll merge :)

[valscion]

This is now in v4.8.0! 🚀