[RFC] peer discovery with mDNS #80

richardschneider · 2018-07-23T07:31:14Z

Background discussion at Future of mDNS discovery

discovery/mdns.md

Stebalien · 2018-07-24T01:50:46Z

discovery/mdns.md

+    <host-name>              AAAA <ipv6 address>
+
+
+Multiple A and AAAA records are expected. The `TXT` is not needed for IPFS peer discovery, but is required by DNS-SD.


So, we'll need it for multiaddrs. We're able to get away with this for now but not once we enable QUIC by default. Ideally, we'd only use TXT records. However, we have to supply an A or AAAA record and also have to supply a port. Personally, I'm all for using our TCP port and real IP addresses when announcing but mostly ignoring them when accepting (unless we get no TXT records in which case we can try the IP/PORT from the other records).

Note: We'd probably want to use dnsaddr=/ip4/.../tcp/... as the format for TXT records.

TXT record dnsaddr=/full/address is a problem. Strings must be less than 128 bytes.

dnsaddr=/ip6/2406:e001:1467:1:ecec:9ca6:XXXX:XXXX/tcp/4009/ipfs/QmTp1fwYnJxc5rUEVM2xowcDiyR61J7yvmBRUQzT4g8MHs almost reaches this limit. If the peer-id hashing algorithm changes to sha2-512 then the byte limit is exceeded.

I strongly suggest that TXT dnsaddr is not used.

Also, we are just trying to discover peers on the local link. So to my thinking is that an IP address, port and peer ID is all that is needed.

Can we send multiple records?

Well, we also have protocols like QUIC and may add other random protocols in the future. We also do need to put the peer ID somewhere.

It looks like the standard way to do this is to break it into multiple records. You can see how DKIM does it here: https://serverfault.com/questions/255580/how-do-i-enter-a-strong-long-dkim-key-into-dns

We could also use a base64 encoded binary multiaddr but I'd rather not.

richardschneider · 2018-07-25T00:07:45Z

@Stebalien

We also do need to put the peer ID somewhere.

The peer id is the first label of the SRV record's name.

richardschneider · 2018-07-25T00:13:25Z

I'm not sure that the spec on TXT record allows multiple strings with the same property name. I know for certain that most "mdns browsers" only show the first/last value,

I'll dig into the specs and report back.

richardschneider · 2018-07-25T00:29:07Z

I was wrong about the maximum string length it is 255 bytes, not 128.

But this does not help with the network MTU. To make life easier, I want all required resource records (in Additional Records) to fit into one message response.

Stebalien · 2018-07-25T01:28:02Z

Yeah, but MTUs are usually at least 1280 bytes. I think we'll be well within that.

discovery/mdns.md

richardschneider · 2018-07-27T00:35:53Z

@Stebalien I hope all you concerns have been answered, dnsaddr=... is now part of the spec.

If you're happy, I start on the worked examples.

richardschneider · 2018-07-27T00:38:05Z

Should we specifically exclude loopback addresses (127.0.0.1 and ::1) from being discovered?

Stebalien · 2018-07-27T20:45:56Z

Should we specifically exclude loopback addresses (127.0.0.1 and ::1) from being discovered?

No, they're actually quite useful for finding multiple nodes on the same machine. However, we do want to avoid advertising addresses on the wrong interfaces. We should also filter incoming advertisements but that gets a bit tricker. At a minimum, we could filter incoming localhost advertisements to localhost.

lidel · 2018-09-17T12:36:53Z

discovery/mdns.md


-`peer-id` is the ID of the peer.  It normally is the base-58 enconding of the hash of the peer's public key.
+`peer-name` is the case-insenstive ID of the peer and less than 64 characters.  It is normally is the base-32 encoding of peer's ID.


I guess until IPNS: should work with case-insensitive identifiers (Base32) ipfs/kubo#5287 is solved this would require apps to do a manual step of converting back to Base58, yes?

What if PeerID is longer than 64 characters? Should we split after 63 characters like proposed in CID as a Subdomain ipfs/in-web-browsers#89 ?

I guess until ipfs/kubo#5287 is solved this would require apps to do a manual step of converting back to Base58, yes?

peer-name is just a unique name, we should not assume it's the peer id. The dnsaddr ends with the peer ID.

What if PeerID is longer than 64 characters? Should we split after 63 characters like noted in ipfs/in-web-browsers#89 ?

I think Workaround A: Split at 63rd character makes sense.

discovery/mdns.md

tomaka · 2018-10-10T15:41:13Z

Ideally a link to theses specs should be added in 4-architecture.md.

richardschneider · 2018-10-11T01:42:17Z

@Stebalien @diasdavid What is needed to get some approval and then merge it?

daviddias · 2018-10-12T10:53:30Z

@richardschneider it would be great to give it a proper review (I can do it probably later next week). One strong case to merge would be if the implementations were both done (probably they are already) and that interoperability has been tested.

tomaka · 2018-10-22T14:37:45Z

@richardschneider it would be great to give it a proper review (I can do it probably later next week). One strong case to merge would be if the implementations were both done (probably they are already) and that interoperability has been tested.

I just tried the latest ipfs/go-ipfs:master Docker image.
Sniffing traffic with Wireshark, it seems to always generate two identical queries in a row, with the service name _ipfs-discovery._udp.local, and never seems to generate any answer, neither from its own queries (as the specs say it should) nor from other queries.

richardschneider · 2018-10-22T23:41:11Z

@tomaka Yes, go-ipfs mdns has been broken for a long time. Background discussion at Future of mDNS discovery.

The "novel" idea here, is that we first write the spec and then implement it!

jacobheun · 2019-04-10T15:51:07Z

The state of this spec looks reasonable to me for us to get this merged in and marked as Draft. rust-libp2p looks to already have implemented this. It would be ideal to get our MDNS implementations in go and js to an interoperable and better state.

richardschneider · 2019-04-24T05:24:38Z

How should a private network interact with MDNS? I think the service-name should change from _p2p._udp.local to _p2p-x.udp.local , where x is the fingerprint of the private network in base16.

raulk · 2019-04-24T11:25:44Z

In the interest of advancing this, I'm listing TODOs based on above comments:

support loopback addresses to enable local process discovery.
context-sensitive address advertisement based on network interface.
private network discovery.

@richardschneider – if you agree on points 1 & 2, do you have enough bandwidth to enhance the spec?

Re: private networks, SGTM unless we want to hide private network membership somehow. But arguably, if that were the case, you shouldn't be using mDNS at all.

Should we enforce that nodes belonging to a pnet only respond to queries for that pnet?

richardschneider · 2019-04-24T12:33:06Z

@raulk

The spec does not prevent loopback, it was just a question.
I think we should postpone "context-sensitive address advertisement based on network interface". My feeling is that a node can filter on TXT dnsaddr already (like swarm filters)
By using a private network specific service-name we get for free that nodes belonging to a pnet only respond to queries for that pnet.

It's midnight my time, I'll update the doc for a private network tomorrow.

raulk · 2019-04-25T10:47:33Z

@richardschneider that's great, thanks!

richardschneider · 2019-05-05T22:51:49Z

@raulk IMHO this is ready to go.

Stebalien

There may still be some open questions but we can fill those in as we discover them in implementation.

LGTM!

(and thanks @richardschneider for sticking this out)

Stebalien · 2019-05-06T22:19:51Z

(i.e., let's just iterate in new PRs so we have something we can work with now)

feat: initial description of peer discovery with MDNS

a4626ef

ghost assigned richardschneider Jul 23, 2018

ghost added the in progress label Jul 23, 2018

richardschneider requested review from daviddias, Stebalien and whyrusleeping July 23, 2018 07:32

richardschneider added 2 commits July 23, 2018 20:15

fix: grammar

9e0e1f5

fix: typos

154424f

Stebalien reviewed Jul 24, 2018

View reviewed changes

docs: multiaddress

b2c450e

gpestana reviewed Jul 25, 2018

View reviewed changes

discovery/mdns.md Outdated Show resolved Hide resolved

richardschneider added 2 commits July 27, 2018 12:01

fix: '_ipfs' => '_p2p'

105fd2b

fix: use TXT dnsaddr for multi address

a9c587c

feat: open issues

2e0219a

lidel mentioned this pull request Aug 6, 2018

Multicast DNS API arewedistributedyet/arewedistributedyet#21

Open

fix: names are case insensitive

8cc1297

lidel reviewed Sep 17, 2018

View reviewed changes

Kubuxu reviewed Oct 9, 2018

View reviewed changes

discovery/mdns.md Outdated Show resolved Hide resolved

vyzo reviewed Oct 10, 2018

View reviewed changes

discovery/mdns.md Show resolved Hide resolved

discovery/mdns.md Outdated Show resolved Hide resolved

doc: long peer names

53d9faf

richardschneider changed the title ~~feat: initial description of peer discovery with MDNS~~ [RFC] peer discovery with MDNS Oct 11, 2018

docs: link to discovery/mdns.md

409c019

docs: few edits to the structure

2f3e55d

ghost assigned daviddias Oct 12, 2018

tomaka mentioned this pull request Oct 27, 2018

Add libp2p-mdns libp2p/rust-libp2p#590

Merged

lidel mentioned this pull request Nov 5, 2018

In-browser discovery of local-network IPFS nodes ipfs/in-web-browsers#45

Open

jacobheun mentioned this pull request Apr 10, 2019

feat: compatibility with go-libp2p-mdns libp2p/js-libp2p-mdns#80

Merged

1 task

jacobheun requested a review from raulk April 10, 2019 15:51

tomaka mentioned this pull request Apr 10, 2019

Trying to talk to legacy js & go IPFS mDNS implementations libp2p/rust-libp2p#968

Closed

2 tasks

lidel mentioned this pull request Apr 22, 2019

Embedded JS-IPFS in Brave (experiment) ipfs/ipfs-companion#716

Closed

52 tasks

editorial refinements.

479f07c

ghost assigned raulk Apr 24, 2019

raulk changed the title ~~[RFC] peer discovery with MDNS~~ [RFC] peer discovery with mDNS Apr 24, 2019

richardschneider mentioned this pull request Apr 25, 2019

Private network vs mDNS richardschneider/net-ipfs-engine#91

Closed

feat: private networks

4c5a459

Stebalien approved these changes May 6, 2019

View reviewed changes

Stebalien merged commit 743b6fd into master May 6, 2019

ghost removed the in progress label May 6, 2019

This was referenced May 6, 2019

discovery: swap mdns lib for grandcat/zeroconf libp2p/go-libp2p#285

Closed

Implement new MDNS spec libp2p/go-libp2p#623

Closed

richardschneider deleted the mdns branch May 6, 2019 23:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] peer discovery with mDNS #80

[RFC] peer discovery with mDNS #80

richardschneider commented Jul 23, 2018

Stebalien Jul 24, 2018

richardschneider Jul 24, 2018

richardschneider Jul 24, 2018 •

edited

Loading

whyrusleeping Jul 24, 2018

Stebalien Jul 24, 2018

richardschneider commented Jul 25, 2018

richardschneider commented Jul 25, 2018

richardschneider commented Jul 25, 2018 •

edited

Loading

Stebalien commented Jul 25, 2018

richardschneider commented Jul 27, 2018

richardschneider commented Jul 27, 2018

Stebalien commented Jul 27, 2018

lidel Sep 17, 2018 •

edited

Loading

richardschneider Sep 17, 2018

tomaka commented Oct 10, 2018

richardschneider commented Oct 11, 2018

daviddias commented Oct 12, 2018

tomaka commented Oct 22, 2018 •

edited

Loading

richardschneider commented Oct 22, 2018

jacobheun commented Apr 10, 2019

richardschneider commented Apr 24, 2019

raulk commented Apr 24, 2019 •

edited by richardschneider

Loading

richardschneider commented Apr 24, 2019

raulk commented Apr 25, 2019

richardschneider commented May 5, 2019

Stebalien left a comment

Stebalien commented May 6, 2019

		<host-name> AAAA <ipv6 address>


		Multiple A and AAAA records are expected. The `TXT` is not needed for IPFS peer discovery, but is required by DNS-SD.


		`peer-id` is the ID of the peer. It normally is the base-58 enconding of the hash of the peer's public key.
		`peer-name` is the case-insenstive ID of the peer and less than 64 characters. It is normally is the base-32 encoding of peer's ID.

[RFC] peer discovery with mDNS #80

[RFC] peer discovery with mDNS #80

Conversation

richardschneider commented Jul 23, 2018

Stebalien Jul 24, 2018

Choose a reason for hiding this comment

richardschneider Jul 24, 2018

Choose a reason for hiding this comment

richardschneider Jul 24, 2018 • edited Loading

Choose a reason for hiding this comment

whyrusleeping Jul 24, 2018

Choose a reason for hiding this comment

Stebalien Jul 24, 2018

Choose a reason for hiding this comment

richardschneider commented Jul 25, 2018

richardschneider commented Jul 25, 2018

richardschneider commented Jul 25, 2018 • edited Loading

Stebalien commented Jul 25, 2018

richardschneider commented Jul 27, 2018

richardschneider commented Jul 27, 2018

Stebalien commented Jul 27, 2018

lidel Sep 17, 2018 • edited Loading

Choose a reason for hiding this comment

richardschneider Sep 17, 2018

Choose a reason for hiding this comment

tomaka commented Oct 10, 2018

richardschneider commented Oct 11, 2018

daviddias commented Oct 12, 2018

tomaka commented Oct 22, 2018 • edited Loading

richardschneider commented Oct 22, 2018

jacobheun commented Apr 10, 2019

richardschneider commented Apr 24, 2019

raulk commented Apr 24, 2019 • edited by richardschneider Loading

richardschneider commented Apr 24, 2019

raulk commented Apr 25, 2019

richardschneider commented May 5, 2019

Stebalien left a comment

Choose a reason for hiding this comment

Stebalien commented May 6, 2019

richardschneider Jul 24, 2018 •

edited

Loading

richardschneider commented Jul 25, 2018 •

edited

Loading

lidel Sep 17, 2018 •

edited

Loading

tomaka commented Oct 22, 2018 •

edited

Loading

raulk commented Apr 24, 2019 •

edited by richardschneider

Loading