Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Faster JSON dumps in format_message with msgspec #1784

Conversation

BuzzCutNorman
Copy link
Contributor

@BuzzCutNorman BuzzCutNorman commented Jun 22, 2023

This is a draft PR to assist discussion and testing related to Issue #1046

msgspec requires python 3.8 and above. PR #2143 once merged will drop support for python 3.7 from then SDK.


📚 Documentation preview 📚: https://meltano-sdk--1784.org.readthedocs.build/en/1784/

@s7clarke10
Copy link
Contributor

Just curious, what happened with the msgspec work to speed up serialization?

@edgarrmondragon
Copy link
Collaborator

Just curious, what happened with the msgspec work to speed up serialization?

I still need to come up with a plan to add this with the least amount of disruption, maybe starting with opt-in.

@BuzzCutNorman
Copy link
Contributor Author

Just curious, what happened with the msgspec work to speed up serialization?

I still need to come up with a plan to add this with the least amount of disruption, maybe starting with opt-in.

@edgarrmondragon, I am happy to help break this PR into smaller PRs that can be merged in any order you would like. Please just let me know how I can assist with the process.

@edgarrmondragon
Copy link
Collaborator

Just curious, what happened with the msgspec work to speed up serialization?

I still need to come up with a plan to add this with the least amount of disruption, maybe starting with opt-in.

@edgarrmondragon, I am happy to help break this PR into smaller PRs that can be merged in any order you would like. Please just let me know how I can assist with the process.

Thanks @BuzzCutNorman! I'll take some time this week to go through this and #2259 to see where it makes sense to split and have a reasonable package layout.

@edgarrmondragon edgarrmondragon force-pushed the 1046-feat-faster-json-dumps-with-msgspec branch 2 times, most recently from 0ec59f2 to 1c7de71 Compare July 17, 2024 03:04
@BuzzCutNorman
Copy link
Contributor Author

@edgarrmondragon I am thinking it is time to close this PR since you have two very good PRs in the works and to be honest this PR is a bit of a mess. WDYT?

@edgarrmondragon
Copy link
Collaborator

I'm happy to close but give you proper credit in what ends up getting merged. Thanks @BuzzCutNorman!

@BuzzCutNorman BuzzCutNorman deleted the 1046-feat-faster-json-dumps-with-msgspec branch August 7, 2024 17:02
@s7clarke10
Copy link
Contributor

I just want to provide personal thanks to you both @BuzzCutNorman and @edgarrmondragon for taking on this piece of work. There will be a lot of benefits from this with a remarkable improvement in the speed of the tap and targets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants