use bitswap sessions when fetching messages, and cancel them

[Stebalien]

1. Explicitly use a session when fetching messages for a block. Technically, GetBlocks would use an internal session so we'd only end up with two, but one per block is even better.
2. Actually cancel sessions after a timeout (threading through the context).

This should reduce the impact of the current bitswap storm a bit.

NOTE: We should seriously consider having a single session for all blocks, but we can do that in a followup as it may have unintended consequences (e.g., leaks).

[Stebalien]

@dirkmc how would you feel about this? I know we used to have issues with not canceling blocks within sessions (unless we cancel the session), but I believe we fixed that. Is using a single long-running session likely to cause issues?

[dirkmc]

I believe Bitswap does send CANCEL for a block whose session has been cancelled: fix: send cancel when GetBlocks() is cancelled

Using a single session to fetch all messages in a single filecoin block makes sense to me 👍

Using a single session to fetch all messages across all incoming filecoin blocks is different in a subtle way.

To help people who aren't as familiar with Bitswap understand the trade-offs:
When a Bitswap session is asked for IPLD blocks:

• it broadcasts a "want-have" request to all connected peers asking if they have the IPLD block
• the peers that respond with HAVE are added to the session
• subsequent requests for IPLD blocks are only sent to peers in the session

So if we use a single session across all incoming filecoin blocks, it will only send requests for IPLD blocks to the peers that are in the session (ie the peers that responded to the initial "who has this IPLD block" message for the first filecoin block after the session started). New peers are only added to the session in the case that a request for a particular IPLD block receives a DONT_HAVE response from all peers: the session will broadcast the request to all connected peers and add any peers that respond with HAVE to the session.

Another consideration is that if several peers have an IPLD block that the local node wants, the session probabilistically chooses which peer to request the block from according to who was first to respond to requests in the past.

So whether to use a single session for all incoming filecoin blocks depends on how stable we think the network will be. If there is a lot of churn we should probably renew the session frequently. On the other hand if there are a ton of peers with the same IPLD block it will probably help to use the same session to get the messages for several filecoin blocks because we will cut down on those initial broadcasted "discovery" messages.

Maybe it would make most sense to add a parameter for how many epochs a bitswap session should last for?

[Stebalien]

I figured we might be better off re-using the same session because:

1. We receive incoming blocks from the same "grafted" pubsub peers.
2. These peers are also going to be fetching the same messages over pubsub.

Basically, I'd expect to always be getting the same messages from the same peers.

But let's stick with the more obviously correct (current) solution for the moment, and experiment later.

[dirkmc]

Yeah that's a good point. I guess what would be really nice would be to make the peer management mechanism in Bitswap pluggable, so that it would simply reflect gossipsub.

[whyrusleeping]

LGTM, also seems fine to have a single session, but in either case, we should land this ASAP