-
-
Notifications
You must be signed in to change notification settings - Fork 487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support reading large stats.json files #423
Support reading large stats.json files #423
Conversation
9f4254c
to
27041e0
Compare
Is there a possibility this feature gets merged @th0r, @valscion? |
Yeah this looks like something that could work! I seem to have forgotten that this PR exists, sorry about that 😳 @henry-alakazhang would you be open to:
We can make do without a test for this as long as the existing tests pass. |
@valscion this seems to be stalled, and it's something that we could make use of for tracking work to produce smaller app bundles when using kiota to build TypeScript SDKs for HTTP APIs. If @henry-alakazhang is not able to progress this contribution I'd be happy to pick it up and move it forward. |
Yeah feel free to pick this change up move it forward @gavinbarron |
Now that I have bandwidth I could also help moving forward with this contribution @valscion, @henry-alakazhang, @gavinbarron. |
Hey whoops sorry, completely missed this. I'll update the PR. |
27041e0
to
e995d16
Compare
|
Updated with latest version of Remains untested as I can't run edit: tests pass locally. also can confirm it still works with a large file |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this looks great and the tests still pass in CI so I'll merge :)
This is now in v4.8.0! 🚀 |
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [webpack-bundle-analyzer](https://github.com/webpack-contrib/webpack-bundle-analyzer) | devDependencies | minor | [`4.7.0` -> `4.8.0`](https://renovatebot.com/diffs/npm/webpack-bundle-analyzer/4.7.0/4.8.0) | --- ### Release Notes <details> <summary>webpack-contrib/webpack-bundle-analyzer</summary> ### [`v4.8.0`](https://github.com/webpack-contrib/webpack-bundle-analyzer/blob/HEAD/CHANGELOG.md#​480) [Compare Source](webpack-contrib/webpack-bundle-analyzer@v4.7.0...v4.8.0) - **Improvement** - Support reading large (>500MB) stats.json files ([#​423](webpack-contrib/webpack-bundle-analyzer#423) by [@​henry-alakazhang](https://github.com/henry-alakazhang)) - Improve search UX by graying out non-matches ([#​554](webpack-contrib/webpack-bundle-analyzer#554) by [@​starpit](https://github.com/starpit)) - **Internal** - Add Node.js v16.x to CI and update GitHub actions ([#​539](webpack-contrib/webpack-bundle-analyzer#539) by [@​amareshsm](https://github.com/amareshsm)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [x] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xMzMuMCIsInVwZGF0ZWRJblZlciI6IjM0LjEzMy4wIn0=--> Co-authored-by: cabr2-bot <[email protected]> Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1783 Reviewed-by: Epsilon_02 <[email protected]> Co-authored-by: Calciumdibromid Bot <[email protected]> Co-committed-by: Calciumdibromid Bot <[email protected]>
Node has a string size limit of 500MB and the current
JSON.parse(fs.readFileSync(...))
logic to readstats.json
files fails for files >500MB.This PR changes the approach for stats parsing to use a stream, similar to this PR for webpack-cli in creating the file: webpack/webpack-cli#2190
I've tested it against a local repo with a 700MB stats.json and it works when it didn't before. I would add a test, but I don't know about adding that large of a file 😂 Could look into generating one if you would like.