PeerStore improvements

[vasco-santos]

PeerStore improvements

Goal: Road to PeerStore v2 and improve browser connectivity

Milestones

Milestone	Issue	PR	State
0) Gather relevant metrics -- testground	TODO	TODO	TODO
1) AddressBook and ProtoBook	N/A	libp2p/js-libp2p#590	DONE
2) Deprecate peerInfo	libp2p/js-libp2p#589	In the tracking issue	DONE
3) Backed Datastore	libp2p/js-libp2p#591	libp2p/js-libp2p#619	DONE
4) keybook and libp2p-keychain	libp2p/js-libp2p#592	libp2p/js-libp2p#634	DONE
5) MetadataBook	NA	libp2p/js-libp2p#638	DONE
6) Multiaddr confidence	TODO	TODO	TODO
7) Peers Priority	TODO	TODO	TODO

Open minor issues

• Uncomment testing public key in keyBook after connection upgraded

Problem

In js-libp2p, we are currently relying on peer-info to track the information of a peer, like its addresses and running protocols. We are moving different PeerInfo objects around, which in some cases are also not properly updated when the identify-push/ identify-delta are triggered, instead of having a single source of truth to handle this metadata, i.e. the PeerStore. After the release of [email protected] (async refactor) we got some improvements on this, but it was just an initial step.

PeerStore provides us several opportunities to improve peers' connectivity and connection management. From peer scoring, to multiaddr confidence, there are several improvements that we can get from a store to manage peers metadata.

Opportunities

Persistence

Centralizing all the information a peer has about its environment enables us to easily persist this data. This will have a big impact on browser nodes, which will be able to connect to the previous peers they were connected to.

With persistence, we will also be able to have Less load on the bootstrap nodes as we can immeditaly connect to other peers without them.

Connection Management / Protocol topology

When js-libp2p reaches a certain treshold of open connections, it needs to trim some connections so that the node continue its healthy operation. PeerStore can provide input to the connection management about its most important peers, since it has information about the protocols each peer runs. This might be an important factor for a peer scoring function combined with the protocol topologies previously introduced.

Dialer efficiency

When js-libp2p dials a peer, it tries to dial several known addresses of a peer in parallel. We use a token based dialer to not use a lot of resources to a single connection. However, we don't have a good way of deciding which subset of addresses we should try first. If we gather metrics from peer addresses stored in the PeerStore, we can create a scoring function for the addresses.

Peer exchange protocol

With a PeerStore with new capabilities, we can also start working on designing and implementing a peer exchange protocol to improve the discovery capabilities of the peers. New discovery mechanisms are really important for browser nodes.

PeerStore components

• AddressBook
• KeyBook
• ProtoBook
• peerMetadataStore?

---

WIP NOTES:

Goals

• Know when multiaddrs are successful and when they fail so we can keep track of their confidence (take aborted dials into account)
  • Understand the confidence in each multiaddr of a peer so we can manage addresses we store/use/exchange
• Manage the priority of peers so that we understand which ones to stay connected to
• Users should be able to have persisted peer stores to short circuit reboots
• Improve the granularity of protocol storage for a peer so that we can keep them up to date

Multiaddr confidence

• If we dial multiple addrs, and we abort the slower addrs, take into account the confidence of the fastest address
  • Update on success, update on failure, aborts are no ops
  • When the Abort is a timeout abort, treat as a failure for all addresses that were tried
    • Need to account for low dial timeouts poisoning potentially valid address confidence
      • Potentialy weight low dial timeout confidence adjustments
• Parameters: latency, availability, last connection timestamp
• Score function

Metrics

Gather metrics on initial startup to get a set of peers connected (bootstrap nodes + browser peers setup with stardust/webrtc-star).

This will allow us to measure average time to have all peers connected, with and without persistence, compare stardust and webrtc-star, among other useful metrics.

Milestones

0. Gather metrics
1. PeerStore structured with AddressBook,ProtoBook
2. Deprecate peerInfo and centralisation of data on PeerStore
3. Backed Datastore {MemDatastore by default}
4. Add keybook and libp2p-keychain
5. Multiaddr confidence + time to leave
6. Peers priority (combined with connMgr)

Relevant references and previous discussions

libp2p/specs#164
libp2p/go-libp2p-core#123
#556

[vasco-santos]

cc @jacobheun @yiannisbot

[vasco-santos]

@hugomrdias do you have any updates on metrics collection via testground? 🙏

[hugomrdias]

Not yet, i only plan to start working on that next week.

[vasco-santos]

I did a small benchmark to get initial metrics, while we do not have testground support for js. Bear in mind that this was a hacky benchmark on my machine and it does not replace proper benchmark once we have the testground.

I created a scenario with a network of 5 peers that discover each other and get connected. Once all the nodes are connected with the other ones, one of the nodes is stopped and started again, so that we can benchmark the difference of having the PeerStore being persisted.

MDNS Discovery (node.js environment)

On 10 different runs, we got the following average results for performing the entire test:

• Without Persistence: ~415ms
• With Persistence: ~336ms

For this case, having a persisted PeerStore was around 1.2x faster to have connectivity with all the other peers. When we are in the wild, with latencies for discovering and communicating with others peers, it is expected that the persisted PeerStore will bring better results, since we do not have to communicate over the wire to get to know peers before trying to connect with them. Moreover, adding more peers to the network should also show more significant improvements on

Webrtc-star Discovery (browser environment)

On 10 different runs, we got the following average results for performing the entire test:

• Without Persistence: ~10250ms
• With Persistence: ~10262ms

Restarting a node with persistence takes longer when persistence is enabled, while the discovery part was the same. Once again, with latencies for discovering and communicating with others peers, it is expected that the persisted PeerStore will bring better results.
We should revisit this when we do not use the star servers as well, since by now we need to establish a connection with the star server before establishing a connection with a peer behind it. Hopefully we will be able to circumvent this issue with the rendezvous and circuit relays and establish both connections in one.

[wemeetagain]

Closed as already implemented