Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Networking] Add a hook for application messages #3759

Merged
merged 6 commits into from
Oct 17, 2024
Merged

Conversation

bfish713
Copy link
Collaborator

@bfish713 bfish713 commented Oct 15, 2024

This PR:

This adds a way for the application layer to send arbitrary messages to other participants via the HotShot networking layer. It's not perfect as it'll do double de/serialization and all messages are fire and forget within hotshot.

This PR does not:

Key places to review:

Changes to SystemContextHandle
Test added to the newtorking tests

@bfish713 bfish713 marked this pull request as ready for review October 15, 2024 12:28
@bfish713 bfish713 requested a review from rob-maron October 15, 2024 12:28
Copy link
Contributor

@ss-es ss-es left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the main change looked good to me, just had some comments

crates/types/src/message.rs Outdated Show resolved Hide resolved
))
.await
.unwrap();

// See whether or not we should be DA
let is_da = node_id < config.da_staked_committee_size as u64;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to take this from config.known_da_nodes instead?

I feel like it would be better to have config.known_da_nodes initialized to the first da_staked_committee_size nodes, and take directly from that here (otherwise we have da nodes getting set independently in two different places, which might lead to surprising behaviour down the line)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we should do this. However the code is like this all over the test suites, I'll create an issue for this one and follow up since it might be a bigger change.

@bfish713 bfish713 requested a review from ss-es October 17, 2024 14:07
Copy link
Contributor

@jparr721 jparr721 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good, but I feel like I don't understand the motivation. Why not just use the external network? Is this basically allowing the application (i.e. sequencer) to talk back to us?

@@ -160,11 +163,14 @@ impl<TYPES: NodeType> NetworkMessageTaskState<TYPES> {

// Handle external messages
MessageKind::External(data) => {
if sender == self.public_key {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this essentially prevent loops with processing messages that we sent?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's basically so we don't broadcast messages to ourselves. I feel like it's unexpected behavior from the applications perspective to get it's own message

Copy link
Contributor

@ss-es ss-es left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@bfish713
Copy link
Collaborator Author

bfish713 commented Oct 17, 2024

This all looks good, but I feel like I don't understand the motivation. Why not just use the external network? Is this basically allowing the application (i.e. sequencer) to talk back to us?

What is the external network in this case? The change is allow the sequencer to communicate through the hotshot layer because it's much more flexible and efficient. For example if a sequencer wanted to get a DA block from another node it sends a web request to a configured url. If non of the configured query nodes have it then it's hosed. Instead the sequencer could use this api to send a request to any HS public key it knows is on the DA committee to get the info.

@bfish713 bfish713 merged commit 86df546 into main Oct 17, 2024
24 checks passed
@bfish713 bfish713 deleted the bf/external-networking branch October 17, 2024 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants