Tunnel improvements

[egbertbouman]

There are still some improvements that I would like to make to the TunnelCommunity in the coming weeks:

• Let peers within the TunnelCommunity announce how they would like to participate in the network. For instance, a node may want to:
  • Relay data
  • Act as an IPv8 and/or uTP exit
  • Be an introduction/rendezvous point for hidden services
• Set up an experiment for donating to the network and obtaining tokens for end-to-end encrypted relaying:
  • Observe operational performance on these machines
  • Tweak and boost the performance
• Since CPU still seems to be the bottleneck, I'd like to make the code a bit more efficient.
  Some examples of what can be improved:
  • Message (de)serialization
  • Before Twisted writes to a socket, it first spends a lot of time in isIPAddress / isIPv6Address
  • Remove isinstance calls from byte counting functions
  • Packet demultiplexing (at Community level)
  • Crypto (also investigate Tunnel crypto improvement with AESGCM #3099)

Function duration breakdown of on_cell:

[devos50]

Should this be assigned to the Tribler 7.3 milestone?

[egbertbouman]

@devos50 Since 7.3 is already in beta, I think this is something for 7.4.

[synctext]

We can hopefully have the doubling of tunnel speed for final Beta of 7.3... Plus trust animation. Then into code freeze and stability only.

[synctext]

ToDo: discuss raw speed, faster download starts, robustness versus security properties such as equal packet size. @egbertbouman

[egbertbouman]

Just a few things that are on my tunnel wishlist:

• Create a live circuit speed test ( bytes/s up and down, including hidden services). I'd like to add some speed test specific messages that allow us to do a speed test without involving a real-world BitTorrent swarm. Using the peer_flags we can extend the tunnel protocol without breaking compatibility.
• Build more endpoints/tools that allow us to gain insight into the inner workings of the tunnel community, especially hidden services. For instance, the status of the PexCommunities, the sizes of hidden swarms, and tunnel packet loss.
• Remove PEER_FLAG_EXIT_ANY, and only use more specific flags: PEER_FLAG_EXIT_IPV8/BT/HTTP. Since the meaning of PEER_FLAG_EXIT_ANY changes as we extend the protocol, we can't really use it to determine which extensions a particular peer supports.
• Switch from cryptography to libnacl, since it seems to be considerably faster. Since we currently use AES-128-GCM, and libnacl only supports AES-256-GCM, this will be a breaking change.
• Add logic to limit the maximum number of hops.