Decentralize BSQ block explorer as we do with other Bisq infrastructure

[wiz]

This is a Bisq Network proposal. Please familiarize yourself with the submission and review process.

Currently Bisq's BSQ explorer is run by a single operator, and it usually works fine. However, like any centralized service running on a single server, it occasionally has issues, such as being out of sync with the network, or simply being offline or unavailable.

Usually when you encounter a malfunctioning block explorer, you can just use a different block explorer, but with BSQ there is only one, so the options are very limited. This is clearly a CPOF of Bisq infrastructure, and we should have more than a single BSQ explorer.

I propose to decentralize the BSQ block explorer infrastructure in the same manner we do with btcnodes, seednodes, and pricenodes. Specifically:

• Split the current explorer operator/maintainer role into 1 x BSQ explorer maintainer role and 4 x BSQ explorer operator roles, similar to how we have with seednodes
• Have the BSQ explorer maintainers be @sqrrm (primary) and @wiz (backup)
• Have the BSQ explorer operators be @Emzy @devinbileck @mrosseel @wiz (same as current seednode operators)
• Have each BSQ explorer run on a unique URL, i.e. explorer1.bisq.network, explorer2.bisq.network, etc.
• Change the current explorer.bisq.network DNS record into a round-robin DNS record to load balance across the 4 explorers
• Add a user preference into Bisq which allow users to choose which BSQ explorer to use, similar to how users can currently select a Bitcoin block explorer (with the round-robin as the default)

Any other ideas how to improve the BSQ explorer infrastructure?

[sqrrm]

It would be great to distribute the explorer. I don't know if we want to run it on the main seed nodes though as it might affect their main purpose of being seed nodes. Better to have a dedicated seed node for each explorer.

The current explorer is, in my opinion, not great. Some work on improving it would go a long way to make the explorer more stable. The API should help.

[cbeams]

• Have each BSQ explorer run on a unique URL, i.e. explorer1.bisq.network, explorer2.bisq.network, etc.

+1 to each explorer running at a unique URL, but from a decentralization perspective, I'd prefer to go with the current scheme, where the operator hosts it under one of their own domains, e.g. as @sqrrm has done with https://explorer.sqrrm.net. It should have some value for the user to see these (i.e. in the list of available explorers), as they'll naturally intuit that these instances are run by individual Bisq contributors. This will have the same effect as seeing the list of seednodes in Bitcoin Core's chainparams.cpp, e.g. seed.bitcoin.sipa.be, seed.bitcoin.jonasschnelli.ch, etc. On the contrary, if all hosts were subdomains under bisq.network, e.g. explorer1, explorer2, the user would get the sense that some kind of more traditional centralized entity is operating them.

• Change the current explorer.bisq.network DNS record into a round-robin DNS record to load balance across the 4 explorers

I've noticed that the two current explorer instances at https://explorer.sqrrm.net and https://explorer.bisq.network can have slightly different data on the main ("Recent") page. At time of this writing they are in sync, but a few minutes prior, one of them had a handful of recent transactions that the other didn't. If the round-robin approach naively load balanced across these instances, the user could have a confusing experience of data appearing and disappearing as they click and/or refresh. Session affinity should solve this, and while implementing it is probably a given, it's perhaps still worth raising awareness about and adding to the requirements explicitly.

[wiz]

I'd prefer to go with the current scheme, where the operator hosts it under one of their own domains, e.g. as @sqrrm has done with https://explorer.sqrrm.net.

You're right, we should use our own domain names as a better indicator of who is running them for trust reasons

If the round-robin approach naively load balanced across these instances, the user could have a confusing experience

Alternatively we could ditch the round-robin idea and randomly select one for the user in the Bisq settings, which they could change manually if they wanted to later on.

[wiz]

Okay setup one for testing at https://bsq.wiz.biz - let's see how it goes, so far I think I see why this is so unstable 😓

[sqrrm]

Great that you set one up. Any ideas on how to make it more stable? It's not my area of expertise.

[wiz]

Yeah, I'll send you a PR soon. The most obvious problem seems to be this weird "delete all the data and re-copy all the data" every time there is any update, as part of the update_data.sh script - I don't see any reason to do this at all, I can simply modify the nginx.conf to serve the JSON files directly from the Bisq seednode's data folder, and there is no reason to copy the data at all.

The second problem which is less obvious, is that nginx serves the site with HTTP headers so the cache time gets set quite high, so if your browser requests a JSON file and it isn't there (very likely because of the first issue) then your browser will cache it as non-existing and not retry to fetch it again later, even if you restart your browser.

[sqrrm]

I introduced the copy of the data for some reason that I can't remember now. Could be the python script had some issue that's no longer an issue. Happy to try again without the copy if it works.

Good find on the caching issue. Always wondered about those missing files, an issue that's more common now with more data.

[wiz]

Ah okay, if the python script needs a stable copy of the data while it indexes it, we can rsync a temporary copy for the indexing... in the meantime, please also take a look at my PR for the new user preference combobox bisq-network/bisq#3792

[wiz]

Closing this proposal as done! 🎉