Collect and re-forward packets received while TpuForwarder is shutting down

[mvines]

This is a follow-up to #2758 which unlocks the test_two_fullnodes_rotate_every_tick_with_transactions multinode test.

When a node is transitioning from validator to leader, the TpuForwarder may have inflight packets that should be retained (just like the BankingStage did when transitioning from leader to validator). The fix is the same, collect all the inflight packets and pass them along once the transition is complete.

This PR contains a slight refactoring of how fullnode manages transitions to:

• Remove duplication between Fullnode::new() and Fullnode::rotate()
• Consolidate the calls to cluster_info::set_leader() that were occurring randomly about the tree into the Tpu, so that it aligns exactly with the state of the transition

Fixes #2675 even more than before.

[codecov]

Codecov Report

Merging #2766 into master will increase coverage by <.1%.
The diff coverage is 70.7%.

@@           Coverage Diff            @@
##           master   #2766     +/-   ##
========================================
+ Coverage    78.7%   78.7%   +<.1%     
========================================
  Files         115     115             
  Lines       19437   19422     -15     
========================================
- Hits        15301   15296      -5     
+ Misses       4136    4126     -10