fix(rpc)!: read from substream while streaming to check for interruptions #3548
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Motivation and Context
Streams should terminate as soon as they are not used, this was a particular problem when
streaming UTXOs to wallets and pruned nodes, when erroring out and retrying, new streams
would be established while the old streams still continued until termination.
The main trick was to read from the yamux stream just before writing to it without blocking if there
was nothing there. This allows yamux to receive the close stream message.
How Has This Been Tested?
New unit test
Manually, added code to forcefully error out of pruned sync after starting the stream. The node had another local node as a forced_sync_peer and so would continuously retry that peer. This caused many (saw up to 30) RPC sessions to accumulate and many concurrent streams. With these changes, the sessions used go between 1 and 0 for that peer.