Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check connected peer weight #213

Merged
merged 6 commits into from
Mar 5, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,13 @@ See the [Building](#building) section for instructions on how to build the relay
- Each subnet API node must have enabled:
- eth API (RPC and WS)
- The P-Chain API node must have enabled:
- info.peers
- platform.getHeight
- platform.validatedBy
- platform.getValidatorsAt OR platform.getCurrentValidators
- If the P-Chain API node is also a subnet validator, it must have enabled:
- The Info API node must have enabled:
- info.peers
- info.getNetworkID
- If the Info API node is also a subnet validator, it must have enabled:
- info.getNodeID
- info.getNodeIP

Expand Down Expand Up @@ -104,14 +106,18 @@ The relayer is configured via a JSON file, the path to which is passed in via th
- The URL of the Avalanche P-Chain API node to which the relayer will connect. This API node needs to have the following methods enabled:
- platform.getHeight
- platform.validatedBy
- platform.getValidatorsAt
- platform.getValidatorsAt OR platform.getCurrentValidators

`"info-api-url": string`

- The URL of the Avalanche Info API node to which the relayer will connect. This API node needs to have the following methods enabled:
- info.peers
- info.getNetworkID

- Additionally, if the Info API node is also a validator, it must have enabled:
- info.getNodeID
- info.getNodeIP

`"storage-location": string`

- The path to the directory in which the relayer will store its state. Defaults to `./awm-relayer-storage`.
Expand Down
26 changes: 9 additions & 17 deletions peers/app_request_network.go
Original file line number Diff line number Diff line change
Expand Up @@ -183,21 +183,15 @@ func NewNetwork(
}

// ConnectPeers connects the network to peers with the given nodeIDs.
// On success, returns the provided set of nodeIDs and a nil error.
// On failure, returns the set of nodeIDs that successfully connected and an error.
func (n *AppRequestNetwork) ConnectPeers(nodeIDs set.Set[ids.NodeID]) (set.Set[ids.NodeID], error) {
// Returns the set of nodeIDs that were successfully connected to.
func (n *AppRequestNetwork) ConnectPeers(nodeIDs set.Set[ids.NodeID]) set.Set[ids.NodeID] {
n.lock.Lock()
defer n.lock.Unlock()

var (
retErr error
trackedNodes set.Set[ids.NodeID]
)

// First, check if we are already connected to all the peers
connectedPeers := n.Network.PeerInfo(nodeIDs.List())
if len(connectedPeers) == nodeIDs.Len() {
return nodeIDs, nil
return nodeIDs
}

// If we are not connected to all the peers already, then we have to iterate
Expand All @@ -212,10 +206,11 @@ func (n *AppRequestNetwork) ConnectPeers(nodeIDs set.Set[ids.NodeID]) (set.Set[i
"Failed to get peers",
zap.Error(err),
)
return nil, err
return nil
}

// Attempt to connect to each peer
var trackedNodes set.Set[ids.NodeID]
for _, peer := range peers {
if nodeIDs.Contains(peer.ID) {
ipPort, err := ips.ToIPPort(peer.PublicIP)
Expand All @@ -225,43 +220,40 @@ func (n *AppRequestNetwork) ConnectPeers(nodeIDs set.Set[ids.NodeID]) (set.Set[i
zap.String("beaconIP", peer.PublicIP),
zap.Error(err),
)
retErr = fmt.Errorf("failed to connect to peers: %v", err)
continue
}
trackedNodes.Add(peer.ID)
n.Network.ManuallyTrack(peer.ID, ipPort)
if len(trackedNodes) == nodeIDs.Len() {
return trackedNodes, retErr
return trackedNodes
}
}
}

// attempt adding api node in case it is a validator
// If the Info API node is in nodeIDs, it will not be reflected in the call to info.Peers.
// In this case, we need to manually track the API node.
if apiNodeID, _, err := n.infoClient.GetNodeID(context.Background()); err != nil {
n.logger.Error(
"Failed to get API Node ID",
zap.Error(err),
)
retErr = fmt.Errorf("failed to get api Node ID: %v", err)
} else if nodeIDs.Contains(apiNodeID) {
if apiNodeIP, err := n.infoClient.GetNodeIP(context.Background()); err != nil {
n.logger.Error(
"Failed to get API Node IP",
zap.Error(err),
)
retErr = fmt.Errorf("failed to get api Node IP: %v", err)
} else if ipPort, err := ips.ToIPPort(apiNodeIP); err != nil {
n.logger.Error(
"Failed to parse API Node IP",
zap.String("nodeIP", apiNodeIP),
zap.Error(err),
)
retErr = fmt.Errorf("failed to parse API Node IP: %v", err)
} else {
trackedNodes.Add(apiNodeID)
n.Network.ManuallyTrack(apiNodeID, ipPort)
}
}

return trackedNodes, retErr
return trackedNodes
}
16 changes: 11 additions & 5 deletions relayer/message_relayer.go
Original file line number Diff line number Diff line change
Expand Up @@ -247,13 +247,19 @@ func (r *messageRelayer) createSignedMessageAppRequest(requestID uint32) (*avala
for node := range nodeValidatorIndexMap {
nodeIDs.Add(node)
}
connectedNodes := r.relayer.network.ConnectPeers(nodeIDs)

// TODO: We may still be able to proceed with signature aggregation even if we fail to connect to some peers.
// We should check if the connected set represents sufficient stake, and continue if so.
_, err = r.relayer.network.ConnectPeers(nodeIDs)
if err != nil {
// Check if we've connected to a stake threshold of nodes
michaelkaplan13 marked this conversation as resolved.
Show resolved Hide resolved
connectedWeight := uint64(0)
for node := range connectedNodes {
connectedWeight += validatorSet[nodeValidatorIndexMap[node]].Weight
}
if connectedWeight/totalValidatorWeight < r.warpQuorum.QuorumNumerator/r.warpQuorum.QuorumDenominator {
r.relayer.logger.Error(
"Failed to connect to peers",
"Failed to connect to a threshold of stake",
zap.Uint64("connectedWeight", connectedWeight),
zap.Uint64("totalValidatorWeight", totalValidatorWeight),
zap.Any("warpQuorum", r.warpQuorum),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

err referenced on lines 268 and 270 is no longer referencing a relevant error.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

zap.Error(err),
)
return nil, err
Expand Down
Loading