Lists of all fediverse instances, Bluesky PDSes, Nostr relays

[snarfed]

These would be useful as hints to which protocol a given URL or domain is on. Right now we have to sniff in Protocol.for_id.

These lists are constantly changing, and some are big, eg the fediverse currently has 20-25k instances, so we'd need some mechanism to regularly fetch and update them. Probably just store each list in a single file on Cloud Storage or similar and load it into memory at startup.

• Fediverse: https://fediverse.observer/ , https://the-federation.info/ , https://fediverse.party/
• Nostr: https://nostr.watch/relays/find , https://stats.nostr.band/ , https://primal.net/explore , https://nashboard.space/ (support Nostr #446)
• Bluesky: TBD

[snarfed]

https://nodes.fediverse.party/ has an easily downloadable list for the fediverse, updated every 6h: https://nodes.fediverse.party/nodes.json

[WisTex]

Will you be able to detect that a server is multi-protocol? For example, Friendica and Hubzilla are both multi-protocol (if the admin has enabled multiple protocols).

[WisTex]

I didn't see it on your list, but there is also https://fedidb.org/

[snarfed]

Yes! FediDB is great. It didn't exist when I originally filed this issue. 😁

[Tamschi]

For most fediverse instances, it should be possible to detect them on demand via NodeInfo for the domain and potentially WebFinger with the URL as parameter.

[snarfed]

Oh sure, for a given domain, definitely. The more interesting question is how to detect newly created instances in general, ie domains that we don't know about at all yet.

[Tamschi]

I don't think we should, for ActivityPub. It doesn't seem relevant to the bridge's functions to do this proactively.

[snarfed]

It would definitely help internally. We often have to figure out which protocol a given URL belongs to, which often requires making an external HTTP fetch. This would often prevent that.

[Tamschi]

Ah, right, but that's much better served with an on-demand cache than with a global list too.

(Arguably, constructing a global or even reasonably complete list for ActivityPub is not possible in the first place.)