Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: introduce ToBytes in bls.h and use it in all spots trivially possible, convert vecBytes to std::array #6515

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

PastaPastaPasta
Copy link
Member

Issue being fixed or feature implemented

we made changes to bls library library to allow std::array based serialization, which resulted in ~10% speedups in serialization in library benchmarks. I noticed today:

pre-this change:
Pasted Graphic

post this change:
Pasted Graphic 1

this shows that roughly 2.5% of total cycles during reindex are spent on vector new in our bls.h object.

What was done?

Convert to array to avoid dynamic allocation slowdown

How Has This Been Tested?

reindexing atm, looks good so far, and performance is improved as expected

Breaking Changes

None

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

…ssible, convert vecBytes to std::array

this results in an improved reindex speed as previously roughly 2% of total cycles were just in allocating the vector.
@PastaPastaPasta PastaPastaPasta added this to the 22.1 milestone Dec 29, 2024
Copy link

coderabbitai bot commented Dec 29, 2024

Walkthrough

The pull request introduces a series of modifications across multiple source files, primarily focusing on the renaming and updating of methods related to byte vector and signature handling. The changes center around the CBLSWrapper and CBLSSignature classes, with a consistent theme of replacing SetByteVector with SetBytes and ToByteVector with ToBytes.

In the src/bls/bls.h file, the CBLSWrapper class sees method signature updates and the addition of new ToBytes methods. The CBLSLazyWrapper class also undergoes changes, with its vecBytes member transitioning from a std::vector<uint8_t> to a std::array<uint8_t, BLSObject::SerSize>, which suggests an optimization in memory management.

The modifications extend to governance and quorum-related files (src/governance/object.cpp, src/governance/vote.cpp, and src/llmq/dkgsession.cpp), where signature-related method calls are updated to use the new naming convention. These changes appear to be part of a broader effort to standardize and potentially improve the handling of cryptographic signatures and byte representations across the codebase.

Tip

CodeRabbit's docstrings feature is now available as part of our Early Access Program! Simply use the command @coderabbitai generate docstrings to have CodeRabbit automatically generate docstrings for your pull request. We would love to hear your feedback on Discord.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
src/llmq/dkgsession.cpp (1)

1024-1026: Consistency for quorumSig manipulation.
The approach parallels the previous signature manipulation, ensuring ToBytes and SetBytes remain consistent for simulation.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6310321 and d350cbc.

📒 Files selected for processing (4)
  • src/bls/bls.h (14 hunks)
  • src/governance/object.cpp (1 hunks)
  • src/governance/vote.cpp (1 hunks)
  • src/llmq/dkgsession.cpp (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • src/governance/object.cpp
🔇 Additional comments (17)
src/governance/vote.cpp (1)

185-185: Use of SetBytes is consistent and appropriate.
This change keeps the code aligned with the new SetBytes approach for handling BLS data.

src/bls/bls.h (15)

62-62: Consistent invocation of SetBytes.
Using SetBytes in the constructor keeps object creation unified with the broader rename from SetByteVector.


Line range hint 106-119: Ensure proper reset on invalid input.
The new SetBytes method correctly resets the wrapper on short or zero-filled input, preventing malformed states.


139-147: Addition of ToBytes methods is a welcome change.
Returning a fixed-size std::array helps eliminate unnecessary dynamic allocations, improving performance.


167-167: Maintain uniform usage of SetBytes.
Replacing direct references to old methods ensures that all code paths use SetBytes consistently.


179-179: Directly writing array bytes is clear.
Using AsBytes with the result of ToBytes is straightforward for serialization.


193-193: De-serialize using SetBytes.
Switching from SetByteVector to SetBytes for reading from streams keeps the routine consistent with the rest of the refactoring.


198-198: Fallback scheme usage is correct.
Attempting the opposite scheme if the first attempt fails is well-handled here, continuing to rely on SetBytes.


218-218: Efficient verification using memcmp.
Comparing the byte arrays via memcmp ensures a direct check of the internal representation for malleability.


234-234: Hex conversion uses ToBytes effectively.
Extracting bytes for hex-string output further demonstrates the benefit of a fixed-size array.


261-264: Fixed-size array in CBLSIdImplicit.
Returning a std::array<uint8_t, 32> for serialization is consistent with the rest of the new design.


397-397: Refactor to fixed-size vecBytes.
Changing from std::vector<uint8_t> to std::array<uint8_t, BLSObject::SerSize> reduces dynamic allocations in CBLSLazyWrapper.


408-408: Default initialization for vecBytes.
Initializing the array with {0} ensures deterministic behavior for an uninitialized buffer.


450-450: Rebuilding vecBytes from ToBytes.
Writing the new array in Serialize is correct and locks in a consistent internal representation.


498-498: Correct use of obj.SetBytes after retrieval.
Deserializing data from vecBytes continues the consistent usage pattern of the new API.


535-535: ToBytes usage in GetHash.
Generating the hash directly from a fixed-size array clarifies the logic and prevents memory reallocation overhead.

src/llmq/dkgsession.cpp (1)

1019-1021: Properly modifying the in-flight signature bytes.
This simulated "lie" operation showcases the flexible usage of ToBytes/SetBytes, but verify that offset [5] is always valid.

@knst
Copy link
Collaborator

knst commented Dec 29, 2024

it conflicts with my PR #6508 which already had one review iteration by Udjin.
Consider reviewing 6508 and rebase this PR on top.

@PastaPastaPasta PastaPastaPasta marked this pull request as draft January 7, 2025 05:44
Copy link

github-actions bot commented Jan 7, 2025

This pull request has conflicts, please rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants