Connection Gating

[aarshkshah1992]

See libp2p/go-libp2p-core#99 for the motivation for this issue.

TODO (PRs to be merged)

• The ConnectionGater interface in core (status: released):
  add connection gating interfaces and types. go-libp2p-core#139

• Use the ConnectionGater in the transport upgrader (status: released):
  call the connection gater when accepting connections and after crypto handshake go-libp2p-transport-upgrader#55

• Use the ConnectionGater in Swarm (status: released):
  implement connection gating support: intercept peer, address dials, upgraded conns go-libp2p-swarm#201

• Shim the Filter to a ConnectionGater (status: done in implement connection gating at the top level #881):
  Shim for Connection Gater go-maddr-filter#22

• Deprecate Filters with ConnectionGater in libp2p and inject it to Swarm and Upgrader. Also, reflection magic to inject gaters into transports (should they need them) (status: merged): implement connection gating at the top level #881

TODO (Work to be done)

• Default Gater implementations.
  • (=> Connection gating: default gater implementations #930)
• Use the Gater in non-upgrade transports like QUIC etc.
  • (=> Connection gating: adapt QUIC transport #931)
• Move the MockConnGater in Swarm to "go-libp2p/p2p/net/mock ". See implement connection gating support: intercept peer, address dials, upgraded conns go-libp2p-swarm#201 (comment).
  • (=> Connection gating: default gater implementations #930)

[willscott]

A timestamp in the GoAway message might not be a bad idea. The UUID is useful to prevent replays, but if the message ends up delayed on a queue / otherwise shows up much in the future, the node receiving it may want to be able to ignore it as out-of-date.

[aarshkshah1992]

As discussed with @raulk offline, we are going to call it the Disconnect protocol instead of the GoAway protocol and designing it will need a separate focused async discussion.

For now, the first win is to get in the connection gating work.

[raulk]

@willscott I don't think I heard about the UUID idea yet -- where can I find more info?

---

Some notes from @aarshkshah1992 and my discussion.

• Is Disconnect an ordinary protocol? Do we mount it as any other protocols? Or is Disconnect a message in a more elemental libp2p wire protocol?

  • We lack a Hello message that advertises capabilities/protocols/metadata.
  • Identify has kind of taken that role in our stack. But it's paradoxical. We pretend it's optional, but it has become rather central to the extent that other protocols now depend on it (e.g. DHT). This is a smell. IMO, it's a sign that we're missing an elemental wire/control protocol.
  • If we decide to go the control protocol path, would we reserve stream 0 for it?

• Disconnect protocol: let's keep it simple. Fields:

  • Reason:
    • over capacity (source: conn manager, or swarm on when connection gater denies inbound connection)
    • temporarily blacklisted (source: adding a peer to a host-wide blacklist)
    • permanently blacklisted (source: adding a peer to a host-wide blacklist)
    • shutting down (source: upon swarm.Close())
    • unused: no streams open (source: conn manager, when a conn has 0 streams)
    • use other connection: we have multiple connections with the same peer, and we're closing one)
    • connection corrupted, e.g. multiplexer corrupted (TODO: how would we even deliver this?)
    • unresponsive: if the peer appears unresponsive (e.g. to a ping message).
    • some places to find inspiration: HTTP status codes, ETH1 rlpx disconnect reasons: https://github.com/ethereum/devp2p/blob/master/rlpx.md#disconnect-0x01.
  • Backoff period: discard; this leaks information, and facilitates attacks.

• Disconnect user API design: TBD.

• Concerning the connection gater, we need to unify it with the Filters, currently an IP-based accept/deny firewall. The connection gater abstraction is a lot more powerful because it can drive functionality like:

  • "do not accept inbound connections until X condition is satisfied", e.g. when a node is starting up and bootsrapping its routing table, blocking inbound connections until the system is warmed up would reduce the exposure to eclipse/poisoning attacks.
  • "outright reject incoming connections if at capacity". Right now we accept all inbound connections and wait for the connmgr to tick and regulate. Let the users choose.

• The Filters somehow found their way into the Upgrader. This is not only conceptually questionable, but it also creates blind spots, e.g. transports that don't use the Upgrader, like QUIC and Bluetooth. Filters won't get applied to them.

• The Upgrader calls the Filters first, before securing the connection and layering the multiplexer. On the one hand, this means that it won't have access to the peer's identity, nor would we be able to send a DISCONNECT message under a stream. On the other hand, it allows us to intercept the connection early (before crypto and muxer negotiation complete), if all we wanted to do is blacklist by IP.

[willscott]

@willscott I don't think I heard about the UUID idea yet -- where can I find more info?

in the previous revision of the OP of this issue

[aarshkshah1992]

@raulk @willscott Apologies, it was a WIP spec that I wanted to refine after the discussion with @raulk and so got rid of it for now to not confuse readers.

[aarshkshah1992]

@raulk

The Filters somehow found their way into the Upgrader. This is not only conceptually questionable, but it also creates blind spots, e.g. transports that don't use the Upgrader, like QUIC and Bluetooth. Filters won't get applied to them.

While the filters wont get applied for QUIC etc. in the upgrader, they will get applied before we incorporate the connection into the Swarm.

func (s *Swarm) addConn(tc transport.CapableConn, dir network.Direction) (*Conn, error) {
	// The underlying transport (or the dialer) *should* filter it's own
	// connections but we should double check anyways.
	raddr := tc.RemoteMultiaddr()
	if s.Filters.AddrBlocked(raddr) {
		tc.Close()
		return nil, ErrAddrFiltered
	}
....

The reason we seem to have injected these filters into the upgrader is to save the cost of upgrading a raw inbound connection as the Swarm can't help us there until connection has been upgraded.

[wenchaopeng]

sorry, may I ask when this feature can be completed?

[raulk]

@wenchaopeng likely this week.

[wenchaopeng]

@raulk received, you are doing a great thing, thank you！

[raulk]

Bulk of the work is completed here, and everything has landed onto corresponding master branches. I'm going to open follow-up issues for each TODO item, so we can close this epic.

[raulk]

This is done, but cannot be released until we implement #931, or we'll cause a regression in the QUIC transport.

[prestonvanloon]

Can we keep this issue open until the feature is released? Otherwise, I am not sure what to track for the availability of connection gating. Thanks!

[aarshkshah1992]

@prestonvanloon That makes sense. The main thing left now before we release this is #931 which should land soon.

[raulk]

@prestonvanloon Released! Read all about it: https://github.com/libp2p/go-libp2p/releases/tag/v0.9.0