Project Flare(decentralised Hole Punching) Phase1 Meta Issue

[aarshkshah1992]

This issue is to aggregate all the work and PRs we have open for taking Hole Punching to completion.

Done criteria: we recommend key consumers like go-ipfs and lotus enable it by default because the functionality works, is well tested, and has quality assertions in place to prevent regressions.

---

Solve Simultaneous connect in multi-stream for TCP hole Punching (Ready for Review)

go-multistream suffers from the simultaneous connect problem wherein it fails when two peers try to connect each other at the same time. This blocks hole punching. This is ONLY a problem for TCP and NOT for QUIC because QUIC does NOT use the multistream protocol for negotiating the security and stream negotiation protocols. The PRs below solve this problem using the method specd out in libp2p/specs#196.

• Implement simultaneous open extension multiformats/go-multistream#42. (released)
• Secure Muxer Interface for Simultaneous Connect go-libp2p-core#180 (released)
• Implement support for simultaneous open go-conn-security-multistream#14. (released)
• Implement support for simultaneous open go-libp2p-transport-upgrader#25. (released)
• Finish and Test the simultaneous connect problem in libp2p peers #1041. (merged)

---

Swarm & Transport changes for Hole-Punching

• add support for transient connections go-libp2p-core#175 (support for transient Conns)
• Treat transient connections as opt-in when opening new streams go-libp2p-swarm#236 (transient conns in Swarm)
• Expose underlying transport connection stat where available go-libp2p-transport-upgrader#71 (upgrader changes for transient conns)
• Context to force a direct dial in the Swarm even if we have a Relayed connection go-libp2p-core#181 (context to force a direct dial) (merged NOT released)
• Support for Hole punching go-libp2p-swarm#233 (force and prefer direct conns)

---

Emit event for NAT device type

• Event for user's NAT Device Type: Tell user if the node is behind an Easy or Hard NAT go-libp2p-core#173 (define event)
• Emit event for User's NAT Type i.e. Hard NAT or Easy NAT #1042 (emit event)

---

Limited Relay Protocol

• Circuit v2 Spec p2p-circuit v2 spec specs#325
• p2p-circuit v2 go-libp2p-circuit#125
• https://github.com/vyzo/libp2p-relay

---

Hole-Punching Co-ordination via Relay Server

• signalling for hole punching #1168 (replaces Signalling for Hole Punching  #1057)
• reject hole punching attempts when we don't have any public addresses #1214.

---

Integrate Limited Relays

• Use circuitv2 code #1183
• add an option to start the relay v2 #1197

---

QUIC Changes

• hole punching support go-libp2p-quic-transport#194

---

AutoRelay

• discover relay v2s using AutoRelay #1351

---

Documentation

• Add documentation about Project Flare (hole punching, Circuit Relay v2, NAT traveral) docs#110

---

Automated testing to prevent regressions

• Basic project flare (hole punching) test  test-plans#21
  • 2022-11-07 note: this item is deemed out of scope since it's, unfortunately a casualty of a time when functionality released without ensuring we have accompanying end-to-end test coverage. Backfiling test coverage for important functionality (including hole punching) is on the libp2p/test-plans roadmap and also dependent on some work in Testground itself. The issue above will get tackled above as part of those efforts.

---

Other

• Solve Local connection address should not be an unspecified address #1024.

The below is a documentation issue that we will do as a best effort.

• NAT Traversal: Tell users if they are behind a symmetric NAT and document UPnP setup and port forwarding on Routers to help users #1017 (document libp2p UPnP and manual port forwarding).

[marten-seemann]

Circuit v2

• p2p-circuit v2 go-libp2p-circuit#125 implements the circuit v2 spec
  • It might need some work to reduce the attack surface.
• Remove v1 circuit relay on the server side. The client has a built-in fallback to v1, and PL will continue to run v1 relays for a while.
• Build logic (using autonat events) to start a v2 relay if the node is publicly accessible.

Tentative timeline: Get this ready for the go-ipfs v0.10 release.

Hole Punching

• QUIC support: hole punching support go-libp2p-quic-transport#194
• libp2p support: Signalling for Hole Punching  #1057
  • reduce the number of retries to 3. Data shows that most connection attempts succeed after one or two attempts, and are unlikely to succeed with more attempts

Tentative timeline: We should defer enabling this until a critical mass of nodes have upgraded and are running v2 relays.

AutoRelay

Once a critical mass of v2 relays has been deployed, rewrite the code to use v2 relays (reserve slots and refresh reservations).
The selection logic should be two-fold:

• Use the DHT and find the closest peers in KAD space, reserve to 2 (or 3) relays from those candidates. Detection of v2 relay support is easy, we can simply wait for identify and see if the node supports the v2 hop protocol.
• Provide an option to configure static v2 relays (without a default list! we want to stop running relays and be a point a centralization) for users who run their own relays.

[marten-seemann]

Closing this issue. Relay v2 and circuit v2 were rolled out with v0.16.0.

[BigLep]

@marten-seemann: I'm viewing the done criteria as meaning "we recommend key consumers like go-ipfs and lotus enable it by default".

I think we should review what that specifically means, but I know in the immediacy we need #1351 .

For being crisp here, can you please add anything to the checklists above that still needs to be done and remove the items that are no longer relevant (e.g., I saw you flag #1017 in #1122 as not relevant)?

To make sure I don't cause any extra churn, we can then have a quick sync to make sure sure we're on the same page. Sound good? Thanks for staying on this trail to its close.

[BigLep]

@marten-seemann : I updated the issue description to include the issues that I believe are open and need to be closed before we call this done:

1. discover relay v2s using AutoRelay #1351
2. Add documentation about Project Flare (hole punching, Circuit Relay v2, NAT traveral) docs#110
3. Basic project flare (hole punching) test  test-plans#21

I attempted to be clear in each issue about what's needed. Feel free to edit/expand.

#1017 was on the existing todo list. Do we need that or can it be removed?

[BigLep]

@marten-seemann @p-shahi : I want to get this closed out. I know we have already agreed to defer the regression test. What is the priority of #1017 ? Should we complete that before marking this as done?

[p-shahi]

Looking at the issue, it seems the remaining work left is to document and add an example for UPnP.
If users are running into issues with hole punching and have requested docs/examples like this, then I say we treat this as high priority.
I'm not sure if that's the case however, so I suggest we close this issue and track 1017 as best effort

[p-shahi]

Moved #1017 to the Best Effort Track and marking this as complete