v1.16: removes outdated check for merkle shreds (backport of #33088) #33136
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an automatic backport of pull request #33088 done by Mergify.
Motivation for v1.16 Backport
ShredFetchStage
implements some basic verification inmodify_packets
. Some of this verification currently relies on having a (root) bank in order to check feature status as well as to check number of slots in an epoch (number of slots in an epoch could be calculated and thenArc<Bank>
could be dropped). The root bank is fetched here:solana/core/src/shred_fetch_stage.rs
Lines 57 to 59 in b13589b
At the time that the bank is fetched, the node just unpacked its' snapshot. So, this initial root bank corresponds to the load-snapshot-slot. Later in the function, we currently use a blocking method to pull shred out of the crossbeam receiver:
solana/core/src/shred_fetch_stage.rs
Line 65 in b13589b
Since we're not getting any packets from TVU forwards, execution is blocked waiting for a shred to show up, and the following code that updates the root bank is never executed:
solana/core/src/shred_fetch_stage.rs
Lines 69 to 70 in b13589b
As a result, the initial root bank at startup (ie the snapshot slot) is held for the duration of the process. Holding this bank is a waste of resources and could interfere with background cleanup that AccountsDB would otherwise be performing.
In #33078, I identified a very similar problem on master with the QUIC receiver. Instead of adjusting the impl, we realized that we could cleanup the code that needed the root bank. Hence, we shipped #33088 to remove outdated feature status check, and then #33105 to remove holding the root bank altogether.
Additional v1.16 Backport
In order to fully fix the bug of the process holding the initial root bank for process lifetime, we will also need a backport of #33105. I plan on pushing that BP after this one lands. I'll also add that I recently performed some separate cleanup on that function in #33066. If we want to optimize for 1-to-1 mapping between master and v1.16 commits, backporting that one first will make #33105 go in cleaner.
Same bug exists on v1.14
Lastly, this same problem exists on v1.14. At the time of writing this, we're on the cusp of upgrading mnb to v1.16 pending some debugging efforts. This isn't a critical bug (ie it will not crash nodes) so I think we can probably skip the v1.14 backport. However, this bank being present will obfuscate other potential debugging efforts such as mem profiling.
Cherry-pick of 1431275 has failed:
To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally
Mergify commands and options
More conditions and actions can be found in the documentation.
You can also trigger Mergify actions by commenting on this pull request:
@Mergifyio refresh
will re-evaluate the rules@Mergifyio rebase
will rebase this PR on its base branch@Mergifyio update
will merge the base branch into this PR@Mergifyio backport <destination>
will backport this PR on<destination>
branchAdditionally, on Mergify dashboard you can:
Finally, you can contact us on https://mergify.com