Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extremely low performance on reading json files generated with ryml::emit_json #301

Closed
D-Alex opened this issue Sep 2, 2022 · 4 comments

Comments

@D-Alex
Copy link

D-Alex commented Sep 2, 2022

Thank you for the great library. We noticed a performance drop of several magnitudes if a bigger file is read, which was generated with ryml::emit_json. The reason is that no line breaks are added, making the parsing painfully slow.

A simple fix could be to change line

this->Writer::_do_write('}');

to

this->Writer::_do_write("}\n");

Performance measurements for parsing a 400KB json file:

  • current implementation: 11014ms
  • after the fix is applied: 22ms --> 500 times speedup
@biojppm
Copy link
Owner

biojppm commented Sep 2, 2022

Thanks! this was an egregious problem that was recently fixed. Which version or commit are you using?

@biojppm
Copy link
Owner

biojppm commented Sep 2, 2022

See #289 and #293

@D-Alex
Copy link
Author

D-Alex commented Sep 2, 2022

Thx for the quick response - We are currently using the header only release rapidyaml-0.4.1.hpp

It seems the already merged fix is a proper fix for this issue as well 👍

@biojppm
Copy link
Owner

biojppm commented Sep 2, 2022

Got it, thanks. Closing this now; feel free to reopen #289 if it resurfaces.

@biojppm biojppm closed this as completed Sep 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants