Add a node-specific ip echo service to remove dependency on ifconfig.co

[mvines]

Problem

1. We use ifconfig.co to figure a node's public IP address
2. The --public-address flag to solana-{fullnode,replicator,gossip} is annoying and error prone. Why does it exist again? Oh, because we use ipconfig.co to figure a node's public IP address.
3. IPv6 address can sneak into gossip because we use ipconfig.co to figure a node's public IP address (IPv6 address in gossip? #4095)

Summary of Changes

• Each node now runs an ifconfig.co-like ip echo service on the TCP port corresponding to the node's gossip UDP port.
• When a node tries to join a cluster it first asks the entrypoint node what its public IP address is. This works for both "private" and "public" clusters

Fixes #936 #4095 #4100

[codecov]

Codecov Report

Merging #4137 into master will decrease coverage by 0.1%.
The diff coverage is 21.2%.

@@           Coverage Diff            @@
##           master   #4137     +/-   ##
========================================
- Coverage    77.8%   77.6%   -0.2%     
========================================
  Files         164     167      +3     
  Lines       28033   28010     -23     
========================================
- Hits        21816   21758     -58     
- Misses       6217    6252     +35

[rob-solana]

This is a step in the right direction: inter-node routes may have to be sensitive to the infra between them. At some point we may have to bite the bullet and do STUN/TURN/WebRTC something?

What's the strategy for this use case: 2 nodes in a NAT form a network, they each look like 192.168.1. to each other, even though they want to be a public cluster. 3rd node from outside that network can't see any of their ports?

[mvines]

At some point we may have to bite the bullet and do STUN/TURN/WebRTC something?

Yeah I think so, but also not super eager to open up that box. Expanding a node's ContactInfo to support multiple IP addresses seems like a nice future improvement too.

What's the strategy for this use case: 2 nodes in a NAT form a network, they each look like 192.168.1. to each other, even though they want to be a public cluster. 3rd node from outside that network can't see any of their ports?

We use this mainly for max TPS testing when we want to ensure all nodes are staying within the cloud provider's local interconnect to avoid network ingress fees that are incurred when using the public IP address. They don't want to be a public cluster, nobody else should attach to the cluster. I think I failed to describe this well above :)

[rob-solana]

At some point we may have to bite the bullet and do STUN/TURN/WebRTC something?

Yeah I think so, but also not super eager to open up that box. Expanding a node's ContactInfo to support multiple IP addresses seems like a nice future improvement too.

I thought about that possibility, but determining route-ability to a peer might be no fun.

What's the strategy for this use case: 2 nodes in a NAT form a network, they each look like 192.168.1. to each other, even though they want to be a public cluster. 3rd node from outside that network can't see any of their ports?

We use this mainly for max TPS testing when we want to ensure all nodes are staying within the cloud provider's local interconnect to avoid network ingress fees that are incurred when using the public IP address. They don't want to be a public cluster, nobody else should attach to the cluster. I think I failed to describe this well above :)

I think I missed this part: how does the bootstrap leader find himself?

[mvines]

I think I missed this part: how does the bootstrap leader find himself?

It needs to be told.

By default it's 127.0.0.1:

solana/fullnode/src/main.rs

Line 196 in f9c0930

[127, 0, 0, 1],

and in a net/-based deployment, we give the bootstrap leader a private or public cloud address (depending on how the testnet was configured) right here:

solana/net/remote/remote-node.sh

Line 87 in f9c0930

--gossip-port "$entrypointIp":8001