Scheduler: Separate Receiving from Scheduling Thread

[apfitzge]

Problem

• New banking stage uses a single thread for both receiving new packets and scheduling
• Time spent receiving packets is time not spent filtering or scheduling packets
• Time spent scheduling is time for more and more packets to get queued in channel from sigverify
• This can lead to high-priority packets sitting in the queue and scheduler thread is not picking them up because it's already begun the scheduling loop

Proposed Solution

• Have a dedicated thread for receiving packets, inserting into a shared data-structure
• I have an implementation, but relies on unsafe code currently. See discord-comment

Some options:

1. The unsafe code is around managing the "map" for storing transaction meta. If we get RuntimeTransaction, I believe we can actually remove the map entirely. That makes this easy to do.
2. Use locks to protect inner elements of "map". Makes the calling code ugly, but gets us to solution faster than waiting for RuntimeTransaction.

[apfitzge]

@tao-stones I posted this on discord a while back. Wonder if you have opinions on which option would be best to take? i.e. wait for runtime transaction, or clean up the intermediate "uglier" solution.

[tao-stones]

runtime-transaction is few steps from action. I'm leaning to wait for it if you can.