Various performance improvements #11
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Improves performance processing pure QUIC traffic by ~20% Relates to zeek/spicy#1565.
There's still a full packet copy within the decrypt_crypto_payload(), but it's one less now.
There should not be a need for the extra copying. hilti::rt::Bytes are mostly std::string and we can pass by const reference as well.
No need for the copy.
As before, avoid unnecessary copies of std::vector instances.
...and return hilti::rt::Bytes directly.
awelzel
force-pushed
the
topic/awelzel/perf-improvements
branch
from
d8c6210 to
6408f4d
Compare
Now that we do not buffer the packet anymore explicitly, we do not need a should_buffer() method.
I suspect the structure here can be improved, but given we're only interested in the form, replace with an anonymous uint8 field.
There's not much point accumulating it in fields if we're never using it, anyhow.
We only need to copy out the buffer, no need to be overly safe.
Think previously we exported all the symbols :-/
We're not actually using any of the fields, so may as well use skip.
bbannier
reviewed
Oct 6, 2023
This removes the iterator usage but removes the explicit copy into std::vector<> in favor of using the hilti::rt::Bytes::data() content directly. Hide the reinterpret_cast<> behind a small helper function. And further feedback from Benjamin.
|
@bbannier - I think I adapted to most you suggested. Mind adding a The top two is the current analyzer version in
It takes ~0.8 seconds for many-requests-12000.pcap and ~1.5 seconds for 16-50000000.pcap without the analyzer enabled. |
? I'll post some hyperfine numbers in a bit, too.
bbannier
approved these changes
Oct 9, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rough summary:
skipoptimization for padding andbytesfields that are otherwise not used.On a pcap created with Python's aioquic package containing roughly 12k quic connections, this PR reduces runtime from ~18.5seconds to ~12seconds. By far the largest impact had removal of the previous
&try / backtrack()approach - see also zeek/spicy#1565.