STABILIZE NETWORK

[cbeams]

Starting in early–mid December, we have experienced a higher rate of stability / reliability issues with Bisq's P2P network.

Problems observed

1. Dropped / delayed arbitration messages. These cases are still relatively rare, but when they occur, individual arbitration messages sent from the trader to the arbitrator, or vice versa, from arbitrator to trader are never received. In one particular case, an entire arbitration ticket never showed up in the arbitrator's queue until days after it was first created. While we have been able to work out every such situation thus far, this is a serious problem as it erodes @bisq-network/arbitrators' confidence as to whether traders are receiving our messages, and in cases of non-responsiveness, it forces us to attempt to contact traders out of band, e.g. via email or via the Bisq forum. This slows the entire process of arbitration down.

2. Dropped trade protocol messages. This appears to be a problem similar to the above, in which certain trade protocol messages, e.g. the message sent when a buyer clicks "Payment Started" is never received by the seller.

It should be emphasized that these problems are the exception and not the rule. The vast majority of trades and arbitration tickets still go smoothly, but any incidence of these issues whatsoever creates real doubt for traders and especially for arbitrators.

Possible causes

1. Recent stability issues on the Tor network itself. @ManfredKarrer has been in communication with Tor developers and confirmed that there have been more problems than usual lately. There is, however, no particular hypothesis as yet about how or why this might be causing the problems described above.
2. Decentralization efforts around seed nodes. We recently attempted to decentralize operation of Bisq's seed nodes, and ran into a number of issues as we did. @ManfredKarrer has taken back operation duties over most or all of our seed nodes, and recently noticed that his nodes on Digital Ocean were experiencing serious issues because of underlying problems with the hosting platform.
3. Accidental partitioning of the network. In 0.6.2 we changed seed nodes' policies around maintaining connections to normal Bisq peers, dropping peers more eagerly in order to free up resources for new ones. This change, and perhaps others, may have led to intermittent partitioning of Bisq's P2P network, such that mailbox messages (stored on seed nodes) never get propagated (or get propagated only after significant delays) to their intended recipients.

[ghost]

This is still a major issue, almost 4 years later. #5526

[ghost]

There are two windows of time when trade protocol messages can be dropped by the Bisq app.

• At startup, during the data download / sync period. (timeframe 30 seconds typ, can be longer)
• During an app-initiated SPV resync. (timeframe 5 to 60 minutes typ.)

The bisq client is divided up into a number of components which subscribe to receive network messages. The network layer delivers messages to all listening components. It does not know what the subscribers are or which messages they are interested in. Subscribers receive all messages but filter to process the ones they are interested in.

The network layer also has no knowledge about whether a particular message was processed or not. So if a particular subscriber is not instantiated at the time of a message destined for it, tough luck -- that message will be dropped.

This is the case of protocol messages during the above mentioned time periods, since bisq infrastructure is not fully started up until after the initial download.

More info in #6785.