Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON parsing slow and memory intense on eval - due to Reflect.setField. #10284

Closed
back2dos opened this issue Jun 2, 2021 · 3 comments
Closed
Assignees
Labels
platform-eval Everything related to the Haxe 4 eval macro interpreter

Comments

@back2dos
Copy link
Member

back2dos commented Jun 2, 2021

Here's an example: https://try.haxe.org/#e820468b

The full data was taken from https://github.com/benmerckx/mime/blob/master/src/mime-db.json (170KB), but on try.haxe that makes the compiler run out of memory (takes about 2s to parse and allocates ~300MB).

For comparison, Context.parse of the same string is about 100x faster.

The reason seems to be that Reflect.setField (called from haxe.format.JsonParser::parseRec) is sloooooow. This can be quickly verified by nooping it out: https://try.haxe.org/#2F9517aC

back2dos added a commit to back2dos/cix that referenced this issue Jun 2, 2021
@Simn Simn self-assigned this Jun 17, 2021
@Simn Simn added the platform-eval Everything related to the Haxe 4 eval macro interpreter label Jun 17, 2021
@Simn
Copy link
Member

Simn commented Jun 17, 2021

The problem is that the object prototype is updated on each setField, and that's not a very fast operation.

I guess for cases like this we really need a proper dictionary mode for objects. This is going to require some engineering, but should be worth the effort.

@Simn
Copy link
Member

Simn commented Jun 17, 2021

I've made a change to put objects in dictionary mode when there's a write-access to an unknown field. This should greatly help dynamic code while not hurt properly typed code much either (it's just one additional branch which I doubt can be measured).

Will keep this issue open so I remember to check the benchmarks because there could be some negative side-effect I'm overlooking.

@RealyUniqueName RealyUniqueName added this to the Backlog milestone Jun 22, 2021
@Simn
Copy link
Member

Simn commented Jul 19, 2021

Benchmarks look good:

brave_GAE7Jbtguc

The date is off but I think that's because that's due to CI/building problems.

@Simn Simn closed this as completed Jul 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform-eval Everything related to the Haxe 4 eval macro interpreter
Projects
None yet
Development

No branches or pull requests

3 participants