Handle SessionLayer disconnection scenarios

[nieznanysprawiciel]

Why:

• The common pattern that contributes to instabilities is cyclic restarting sessions with Nodes. We would like to minimize situations when we disconnect from other Node
• On the other side sometimes Nodes think, that they have session established with us, when in reality they don't. In this case they attempt to send packets and we are dropping them. If they are using unreliable transport, they will never find out. If they are using tcp, they have to wait for tcp timeout after not getting packets confirmations

What:

• Consider adding Reason field to Disconnected protocol message
  • Consider what variants of Reason should we introduce. Proposals:
    • Reason::Shutdown - sender is shutting down
    • Reason::RestartSession - sender doesn't know the session. This would allow receiver to reestablish session gracefully without breaking communication (Distinguishes situation when we are sending disconnect, because we are disappearing from the network)
    • Reason::ProtocolError - when establishing session sender got incorrect/unexpected protocol message and will not continue session initialization. This can gracefully break session attempt without need to wait for timeout on other party side. (Error could be more detailed)
  • Adding fields to protobuf is by definition backward compatible. New Nodes will understand Reason field, old Nodes will just ignore it
• Go through all Disconnected occurrences and check if they serve the purpose of improving session stability
  • Check if we don't get into Disconnected loop when Nodes send this message in response to previous Disconnected
  • Check if there are any other places where sending Disconnected would be beneficial
  • Validate session initialization code if we send Disconnected where necessary
• Test disconnection scenarios
  • Send Disconnectedin different phases of initialization and check if the code is behaving properly (leaving consistent state, aborting initialization futures etc)