Improve trim speed during XML parsing. #1216
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Use native string.trim instead of regex replace for performance. remove log line. fix quotes in tests.
shellicar
force-pushed
the
fix/trim-speed
branch
from
16ee561
to
499395a
Compare
frebos88
approved these changes
May 3, 2023
cgerikj
approved these changes
May 4, 2023
|
Tested the PR out and also saw a huge improvement in performance. 👌 |
|
Just wondering if this project is being maintained anymore? Another PR that seems quite important and supported isn't merged either (another issue I just ran into which brought me back here) - #1212 I ended up just forking the project and hosting it on my own package feed, but I figured there would have been some activity in the past few months. |
|
Good one, thanks for submitting it, I will include in the next release. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue with parsing large XML responses
I recently noticed an issue with slow parsing of large (1-2mb+) responses.
I narrowed the issue down to the trim(text) method in wdsl/index.ts
This uses regex to remove whitespace characters at the start and end of a string.
This has performance implications when the string is very long, as I believe the trimRight regex needs to scan the entire content before reaching the end and then backtrack.
This issue was so bad it was causing requests to timeout after (10+ minutes).
Solution
I couldn't see any other tests testing the whitespace removal, so I added some.
I validated that it was extremely slow before the change (I gave up waiting after around 5 minutes).
I validated that after the change it was fast (~10ms).
And that the expected output is the same.