-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use a faster JSON library #6143
Conversation
Codecov Report
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great idea, I've heard good things about orjson
too, and I agree adoption is growing (eg see FastAPI).
Quick idea for converting this to a base
import for frictionless usage in other integrations…
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice stuff!
Motivation
We parse JSON from multiple endpoints, with each payload being many MiB in size. Some users in resource constrained environments are experiencing abnormally high CPU usage. A more efficient parser should ameliorate that.
Additional Notes
orjson is currently the fastest JSON library available for Python, while also being the least resource-intensive (especially regarding memory). It achieves this by virtue of being a light wrapper around Rust's https://serde.rs
Other Python projects are beginning to recommend and use it too
See: https://github.com/ijl/orjson#performance