Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: report changes in supported protocols to ConnectionHandler #3651

Merged
merged 61 commits into from
May 8, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
f2801fb
Report changes in supported protocols back to `ConnectionHandler`
thomaseizinger Mar 20, 2023
9a28cd2
Allow reporting of supported protocols by remote
thomaseizinger Mar 22, 2023
e536dea
Consume supported protocols in identify
thomaseizinger Mar 24, 2023
7699a1e
Report a remote's protocols to other handlers
thomaseizinger Mar 24, 2023
450fc1e
Add test for kademlia client mode
thomaseizinger Mar 24, 2023
a972b9c
Implement kademlia client-mode
thomaseizinger Mar 24, 2023
f9cf33f
Add tests for two servers connecting
thomaseizinger Mar 24, 2023
deb30f3
Report additions and removals of protocols instead
thomaseizinger Apr 13, 2023
3c1bf5f
Report listen protocols on startup to connection
thomaseizinger Apr 13, 2023
9e70729
Introduce `SupportedProtocols` type
thomaseizinger Apr 13, 2023
f1328c5
Deduplicate code and propagate supported protocols only once
thomaseizinger Apr 19, 2023
a4fbcda
Extend docs
thomaseizinger Apr 19, 2023
3e33108
Merge branch 'master' into 2680-explore
thomaseizinger Apr 19, 2023
572ed90
Fix compile errors
thomaseizinger Apr 19, 2023
586394b
Report changes to listen/external address set
thomaseizinger Apr 26, 2023
b329a09
Simplify identify protocol
thomaseizinger Apr 26, 2023
a63af89
Check for changes in inbound protocols on every poll
thomaseizinger Apr 26, 2023
72510dd
Fix clippy lints
thomaseizinger Apr 26, 2023
8ffafdd
Report changes in `SupportedProtocols`
thomaseizinger Apr 26, 2023
ea1a087
Only report changes to handler if there actually was a change
thomaseizinger Apr 26, 2023
27bc507
Do not implicitly dial peers upon push
thomaseizinger Apr 27, 2023
58039d9
Merge branch 'master' into 2680-explore
thomaseizinger Apr 27, 2023
74dd94a
Remove unused import
thomaseizinger Apr 27, 2023
628b519
Fix compile error
thomaseizinger Apr 27, 2023
c7b5011
Remove dbg!
thomaseizinger Apr 27, 2023
a02ca55
Update swarm/src/handler.rs
thomaseizinger Apr 27, 2023
c347c8a
Merge branch '2680-explore' of github.com:libp2p/rust-libp2p into 268…
thomaseizinger May 2, 2023
f3e5e71
Combine match arms where possible
thomaseizinger May 2, 2023
b7fa7ef
Add comment explaining static hashset
thomaseizinger May 2, 2023
2bd9d73
Add docs
thomaseizinger May 2, 2023
e90c40d
Update supported protocols for push messages
thomaseizinger May 2, 2023
84979e4
Use `pop` to avoid panicking branch in `remove`
thomaseizinger May 2, 2023
dbfc7e7
Use `poll_unpin`
thomaseizinger May 2, 2023
f2d2c88
Use `if let` for consistency
thomaseizinger May 2, 2023
b41aeb8
Rewrite to `if let` for consistency
thomaseizinger May 2, 2023
bcd872b
Fill in todo in toggle
thomaseizinger May 2, 2023
021f1d4
Merge branch 'master' into 2680-explore
thomaseizinger May 2, 2023
fb096ad
Merge branch 'master' into 2680-explore
thomaseizinger May 2, 2023
e95c738
Merge branch 'master' into 2680-explore
thomaseizinger May 4, 2023
82642b8
Restore `libp2p-kad`
thomaseizinger May 4, 2023
eb66489
Fix formatting
thomaseizinger May 4, 2023
8c47bd6
Fix unit tests
thomaseizinger May 4, 2023
bf9421e
Change test to assert actual events received
thomaseizinger May 4, 2023
a82343a
Don't report empty set of protocols to handler
thomaseizinger May 4, 2023
19cd9b9
Use constructor
thomaseizinger May 4, 2023
6d3e9ee
Introduce test helper
thomaseizinger May 4, 2023
df93a4e
Make tests less noisy
thomaseizinger May 4, 2023
c50bcfd
Further simplify test and add comments
thomaseizinger May 4, 2023
a799798
Add failing test
thomaseizinger May 4, 2023
0260ad1
Make test pass, i.e. only report actual changes back to the handler
thomaseizinger May 4, 2023
bf99654
Extract helpers to make fields crate-private and add docs
thomaseizinger May 4, 2023
46f4e96
Return `SmallVec` from `from_full_sets` which allows iteration
thomaseizinger May 4, 2023
ae7fc93
Fix clippy warnings
thomaseizinger May 4, 2023
fe9a6e3
Fix rustdoc
thomaseizinger May 4, 2023
d7651b4
Merge branch 'master' into 2680-explore
thomaseizinger May 8, 2023
75681c1
Add changelog entry
thomaseizinger May 8, 2023
bdfb04f
Merge branch 'master' into 2680-explore
thomaseizinger May 8, 2023
88362e5
Sort imports
thomaseizinger May 8, 2023
3c8d326
Merge branch 'master' into 2680-explore
thomaseizinger May 8, 2023
77e4e5b
Merge branch 'master' into 2680-explore
thomaseizinger May 8, 2023
5a772e1
Merge branch 'master' into 2680-explore
thomaseizinger May 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 47 additions & 15 deletions protocols/identify/src/handler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,15 @@ use libp2p_identity::PeerId;
use libp2p_identity::PublicKey;
use libp2p_swarm::handler::{
ConnectionEvent, DialUpgradeError, FullyNegotiatedInbound, FullyNegotiatedOutbound,
ProtocolsChange,
ProtocolSupport, ProtocolsChange,
};
use libp2p_swarm::{
ConnectionHandler, ConnectionHandlerEvent, ConnectionHandlerUpgrErr, KeepAlive,
NegotiatedSubstream, SubstreamProtocol,
};
use log::warn;
use smallvec::SmallVec;
use std::collections::VecDeque;
use std::collections::{HashSet, VecDeque};
use std::{io, pin::Pin, task::Context, task::Poll, time::Duration};

/// Protocol handler for sending and receiving identification requests.
Expand Down Expand Up @@ -85,7 +85,8 @@ pub struct Handler {
/// Address observed by or for the remote.
observed_addr: Multiaddr,

local_supported_protocols: Vec<String>,
local_supported_protocols: HashSet<String>,
remote_supported_protocols: HashSet<String>,
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
}

/// An event from `Behaviour` with the information requested by the `Handler`.
Expand Down Expand Up @@ -138,7 +139,8 @@ impl Handler {
protocol_version,
agent_version,
observed_addr,
local_supported_protocols: vec![],
local_supported_protocols: HashSet::new(),
remote_supported_protocols: HashSet::new(),
}
}

Expand Down Expand Up @@ -187,10 +189,34 @@ impl Handler {
) {
match output {
future::Either::Left(remote_info) => {
self.events
.push(ConnectionHandlerEvent::ReportRemoteProtocols {
protocols: remote_info.protocols.clone(),
});
let new_remote_protocols = HashSet::from_iter(remote_info.protocols.clone());

let remote_added_protocols = new_remote_protocols
.difference(&self.remote_supported_protocols)
.cloned()
.collect::<HashSet<_>>();
let remote_removed_protocols = self
.remote_supported_protocols
.difference(&new_remote_protocols)
.cloned()
.collect::<HashSet<_>>();

if !remote_added_protocols.is_empty() {
self.events
.push(ConnectionHandlerEvent::ReportRemoteProtocols(
ProtocolSupport::Added(remote_added_protocols),
));
}

if !remote_removed_protocols.is_empty() {
self.events
.push(ConnectionHandlerEvent::ReportRemoteProtocols(
ProtocolSupport::Removed(remote_removed_protocols),
));
}

self.remote_supported_protocols = new_remote_protocols;

self.events
.push(ConnectionHandlerEvent::Custom(Event::Identified(
remote_info,
Expand Down Expand Up @@ -251,7 +277,7 @@ impl ConnectionHandler for Handler {
protocol_version: self.protocol_version.clone(),
agent_version: self.agent_version.clone(),
listen_addrs,
protocols: self.local_supported_protocols.clone(),
protocols: Vec::from_iter(self.local_supported_protocols.clone()),
observed_addr: self.observed_addr.clone(),
};

Expand Down Expand Up @@ -311,10 +337,11 @@ impl ConnectionHandler for Handler {
self.inbound_identify_push.take();

if let Ok(info) = res {
self.events
.push(ConnectionHandlerEvent::ReportRemoteProtocols {
protocols: info.protocols.clone(),
});
// TODO: report new protocols
// self.events
// .push(ConnectionHandlerEvent::ReportRemoteProtocols {
// protocols: info.protocols.clone(),
// });
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open TODO item.

return Poll::Ready(ConnectionHandlerEvent::Custom(Event::Identified(info)));
}
}
Expand Down Expand Up @@ -353,8 +380,13 @@ impl ConnectionHandler for Handler {
self.on_dial_upgrade_error(dial_upgrade_error)
}
ConnectionEvent::AddressChange(_) | ConnectionEvent::ListenUpgradeError(_) => {}
ConnectionEvent::LocalProtocolsChange(ProtocolsChange { protocols }) => {
self.local_supported_protocols = protocols.to_vec();
ConnectionEvent::LocalProtocolsChange(ProtocolsChange::Added(added)) => {
self.local_supported_protocols.extend(added.cloned());
}
ConnectionEvent::LocalProtocolsChange(ProtocolsChange::Removed(removed)) => {
for p in removed {
self.local_supported_protocols.remove(p);
}
}
ConnectionEvent::RemoteProtocolsChange(_) => {}
}
Expand Down
50 changes: 32 additions & 18 deletions protocols/kad/src/handler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ use libp2p_swarm::{
NegotiatedSubstream, SubstreamProtocol,
};
use log::trace;
use std::collections::VecDeque;
use std::collections::{HashSet, VecDeque};
use std::task::Waker;
use std::{
error, fmt, io, marker::PhantomData, pin::Pin, task::Context, task::Poll, time::Duration,
Expand Down Expand Up @@ -86,6 +86,8 @@ pub struct KademliaHandler<TUserData> {

/// The current state of protocol confirmation.
protocol_status: ProtocolStatus,

remote_supported_protocols: HashSet<String>,
}

/// The states of protocol confirmation that a connection
Expand Down Expand Up @@ -503,6 +505,7 @@ where
requested_streams: Default::default(),
keep_alive,
protocol_status: ProtocolStatus::Unknown,
remote_supported_protocols: Default::default(),
}
}

Expand Down Expand Up @@ -790,26 +793,37 @@ where
ConnectionEvent::AddressChange(_)
| ConnectionEvent::ListenUpgradeError(_)
| ConnectionEvent::LocalProtocolsChange(_) => {}
ConnectionEvent::RemoteProtocolsChange(ProtocolsChange { protocols }) => {
// TODO: We should cache this / it will get simpler with #2831.
let kademlia_protocols = self
.config
.protocol_config
.protocol_names()
.iter()
.filter_map(|b| String::from_utf8(b.to_vec()).ok())
.collect::<Vec<_>>();

let remote_supports_our_kademlia_protocols =
kademlia_protocols.iter().all(|p| protocols.contains(p));

if remote_supports_our_kademlia_protocols {
self.protocol_status = ProtocolStatus::Confirmed;
} else {
self.protocol_status = ProtocolStatus::NotSupported;
ConnectionEvent::RemoteProtocolsChange(ProtocolsChange::Added(added)) => {
for p in added {
self.remote_supported_protocols.insert(p.to_owned());
}
}
ConnectionEvent::RemoteProtocolsChange(ProtocolsChange::Removed(removed)) => {
for p in removed {
self.remote_supported_protocols.remove(p);
}
}
}

// TODO: We should cache this / it will get simpler with #2831.
let our_kademlia_protocols = self
.config
.protocol_config
.protocol_names()
.iter()
.filter_map(|b| String::from_utf8(b.to_vec()).ok())
.collect::<Vec<_>>();

let remote_supports_our_kademlia_protocols = self
.remote_supported_protocols
.iter()
.any(|p| our_kademlia_protocols.contains(p));

if remote_supports_our_kademlia_protocols {
self.protocol_status = ProtocolStatus::Confirmed;
} else {
self.protocol_status = ProtocolStatus::NotSupported;
}
}
}

Expand Down
91 changes: 70 additions & 21 deletions swarm/src/connection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ pub use error::{

use crate::handler::{
AddressChange, ConnectionEvent, ConnectionHandler, DialUpgradeError, FullyNegotiatedInbound,
FullyNegotiatedOutbound, ListenUpgradeError, ProtocolsChange,
FullyNegotiatedOutbound, ListenUpgradeError, ProtocolSupport, ProtocolsAdded, ProtocolsChange,
ProtocolsRemoved,
};
use crate::upgrade::{InboundUpgradeSend, OutboundUpgradeSend, SendWrapper, UpgradeInfoSend};
use crate::{ConnectionHandlerEvent, ConnectionHandlerUpgrErr, KeepAlive, SubstreamProtocol};
Expand All @@ -45,6 +46,7 @@ use libp2p_core::upgrade::{InboundUpgradeApply, OutboundUpgradeApply};
use libp2p_core::Endpoint;
use libp2p_core::{upgrade, ProtocolName as _, UpgradeError};
use libp2p_identity::PeerId;
use std::collections::HashSet;
use std::future::Future;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::task::Waker;
Expand Down Expand Up @@ -146,7 +148,7 @@ where
SubstreamRequested<THandler::OutboundOpenInfo, THandler::OutboundProtocol>,
>,

supported_protocols: Vec<String>,
supported_protocols: HashSet<String>,
}

impl<THandler> fmt::Debug for Connection<THandler>
Expand Down Expand Up @@ -184,7 +186,7 @@ where
substream_upgrade_protocol_override,
max_negotiating_inbound_streams,
requested_substreams: Default::default(),
supported_protocols: vec![],
supported_protocols: HashSet::new(),
}
}

Expand Down Expand Up @@ -217,6 +219,21 @@ where
supported_protocols,
} = self.get_mut();

let protocol = handler.listen_protocol();

let new_protocols = protocol
.upgrade()
.protocol_info()
.filter_map(|i| String::from_utf8(i.protocol_name().to_vec()).ok())
.collect::<HashSet<_>>();

handler.on_connection_event(ConnectionEvent::LocalProtocolsChange(
ProtocolsChange::Added(ProtocolsAdded {
protocols: new_protocols.difference(&HashSet::new()).peekable(),
}),
));
*supported_protocols = new_protocols;
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved

loop {
match requested_substreams.poll_next_unpin(cx) {
Poll::Ready(Some(Ok(()))) => continue,
Expand Down Expand Up @@ -248,11 +265,23 @@ where
Poll::Ready(ConnectionHandlerEvent::Close(err)) => {
return Poll::Ready(Err(ConnectionError::Handler(err)));
}
Poll::Ready(ConnectionHandlerEvent::ReportRemoteProtocols { protocols }) => {
Poll::Ready(ConnectionHandlerEvent::ReportRemoteProtocols(
ProtocolSupport::Added(protocols),
)) => {
handler.on_connection_event(ConnectionEvent::RemoteProtocolsChange(
ProtocolsChange {
protocols: &protocols,
},
ProtocolsChange::Added(ProtocolsAdded {
protocols: protocols.difference(&HashSet::new()).peekable(), // This is a bit of a hack to use the same type internally in `ProtocolsAdded`.
}),
));
continue;
}
Poll::Ready(ConnectionHandlerEvent::ReportRemoteProtocols(
ProtocolSupport::Removed(protocols),
)) => {
handler.on_connection_event(ConnectionEvent::RemoteProtocolsChange(
ProtocolsChange::Removed(ProtocolsRemoved {
protocols: protocols.difference(&HashSet::new()).peekable(), // This is a bit of a hack to use the same type internally in `ProtocolsRemoved`.
}),
));
continue;
}
Expand Down Expand Up @@ -367,21 +396,33 @@ where
Poll::Ready(substream) => {
let protocol = handler.listen_protocol();

let mut new_protocols = protocol
let new_protocols = protocol
.upgrade()
.protocol_info()
.filter_map(|i| String::from_utf8(i.protocol_name().to_vec()).ok())
.collect::<Vec<_>>();

new_protocols.sort();

if supported_protocols != &new_protocols {
handler.on_connection_event(ConnectionEvent::LocalProtocolsChange(
ProtocolsChange {
protocols: &new_protocols,
},
));
*supported_protocols = new_protocols;
.collect::<HashSet<_>>();

if &new_protocols != supported_protocols {
let mut added_protocols =
new_protocols.difference(supported_protocols).peekable();
let mut removed_protocols =
supported_protocols.difference(&new_protocols).peekable();

if added_protocols.peek().is_some() {
handler.on_connection_event(ConnectionEvent::LocalProtocolsChange(
ProtocolsChange::Removed(ProtocolsRemoved {
protocols: added_protocols,
}),
));
}

if removed_protocols.peek().is_some() {
handler.on_connection_event(ConnectionEvent::LocalProtocolsChange(
ProtocolsChange::Removed(ProtocolsRemoved {
protocols: removed_protocols,
}),
));
}
}

negotiating_in.push(SubstreamUpgrade::new_inbound(substream, protocol));
Expand Down Expand Up @@ -956,8 +997,16 @@ mod tests {
Self::OutboundOpenInfo,
>,
) {
if let ConnectionEvent::LocalProtocolsChange(ProtocolsChange { protocols }) = event {
self.reported_protocols = protocols.to_vec();
match event {
ConnectionEvent::LocalProtocolsChange(ProtocolsChange::Added(added)) => {
self.reported_protocols.extend(added.cloned());
}
ConnectionEvent::LocalProtocolsChange(ProtocolsChange::Removed(removed)) => {
for protocol in removed {
self.reported_protocols.retain(|p| p != protocol);
}
}
_ => {}
}
}

Expand Down
Loading