Add transaction index in slot to geyser plugin TransactionInfo #25688
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CriesofCarrots
force-pushed
the
add-tx-index
branch
4 times, most recently
from
e64f036 to
55867b6
Compare
CriesofCarrots
force-pushed
the
add-tx-index
branch
from
55867b6 to
d44da7e
Compare
CriesofCarrots
changed the title
CriesofCarrots
force-pushed
the
add-tx-index
branch
from
d44da7e to
68df61f
Compare
carllin
reviewed
Jun 9, 2022
lijunwangs
reviewed
Jun 9, 2022
CriesofCarrots
force-pushed
the
add-tx-index
branch
5 times, most recently
from
f07899b to
f1fb849
Compare
lijunwangs
reviewed
Jun 21, 2022
Codecov Report
@@ Coverage Diff @@
## master #25688 +/- ##
========================================
Coverage 81.9% 81.9%
========================================
Files 631 631
Lines 174252 174388 +136
========================================
+ Hits 142728 142890 +162
+ Misses 31524 31498 -26 |
carllin
reviewed
Jun 22, 2022
added 13 commits
June 22, 2022 13:11
CriesofCarrots
force-pushed
the
add-tx-index
branch
from
bdaf8de to
853b61b
Compare
|
All right, it's go time. Happy to do follow-ups, if you think of anything else, @carllin |
mergify bot
pushed a commit
that referenced
this pull request
Jun 23, 2022
* Define shuffle to prep using same shuffle for multiple slices * Determine transaction indexes and plumb to execute_batch * Pair transaction_index with transaction in TransactionStatusService * Add new ReplicaTransactionInfoVersion * Plumb transaction_indexes through BankingStage * Prepare BankingStage to receive transaction indexes from PohRecorder * Determine transaction indexes in PohRecorder; add field to WorkingBank * Add PohRecorder::record unit test * Only pass starting_transaction_index around PohRecorder * Add helper structs to simplify test DashMap * Pass entry and starting-index into process_entries_with_callback together * Add tx-index checks to test_rebatch_transactions * Revert shuffle definition and use zip/unzip * Only zip/unzip if randomize * Add confirm_slot_entries test * Review nits * Add type alias to make sender docs more clear (cherry picked from commit a6ba5a9) # Conflicts: # core/src/banking_stage.rs # ledger/src/blockstore_processor.rs # poh/src/poh_recorder.rs
gregcusack
pushed a commit
to gregcusack/solana
that referenced
this pull request
Jun 23, 2022
…a-labs#25688) * Define shuffle to prep using same shuffle for multiple slices * Determine transaction indexes and plumb to execute_batch * Pair transaction_index with transaction in TransactionStatusService * Add new ReplicaTransactionInfoVersion * Plumb transaction_indexes through BankingStage * Prepare BankingStage to receive transaction indexes from PohRecorder * Determine transaction indexes in PohRecorder; add field to WorkingBank * Add PohRecorder::record unit test * Only pass starting_transaction_index around PohRecorder * Add helper structs to simplify test DashMap * Pass entry and starting-index into process_entries_with_callback together * Add tx-index checks to test_rebatch_transactions * Revert shuffle definition and use zip/unzip * Only zip/unzip if randomize * Add confirm_slot_entries test * Review nits * Add type alias to make sender docs more clear
gregcusack
pushed a commit
that referenced
this pull request
Jun 29, 2022
* add three gossip metrics measuring gossip loop times * add 5 metrics * rm space * rm space * Update SECURITY.md - fix nav link - add bounty split policy for duplicate reports * Add transaction index in slot to geyser plugin TransactionInfo (#25688) * Define shuffle to prep using same shuffle for multiple slices * Determine transaction indexes and plumb to execute_batch * Pair transaction_index with transaction in TransactionStatusService * Add new ReplicaTransactionInfoVersion * Plumb transaction_indexes through BankingStage * Prepare BankingStage to receive transaction indexes from PohRecorder * Determine transaction indexes in PohRecorder; add field to WorkingBank * Add PohRecorder::record unit test * Only pass starting_transaction_index around PohRecorder * Add helper structs to simplify test DashMap * Pass entry and starting-index into process_entries_with_callback together * Add tx-index checks to test_rebatch_transactions * Revert shuffle definition and use zip/unzip * Only zip/unzip if randomize * Add confirm_slot_entries test * Review nits * Add type alias to make sender docs more clear * Update SECURITY.md finish filling out the table.... * rpc: fix possible deadlock in rpc (#26051) * Add StatusCache::root_slot_deltas() and use it (#26170) * Remove InMemAccountsIndex::map() and use map_internal directly (#26189) * [quic]Decrement total_streams correctly (#26158) * remove comment * alphabetical metrics. no abbreviations * remove trailing white space * cargo fmt to update code format/readability Co-authored-by: Trent Nelson <[email protected]> Co-authored-by: Tyera Eulberg <[email protected]> Co-authored-by: Boqin Qin(秦 伯钦) <[email protected]> Co-authored-by: Brooks Prumo <[email protected]> Co-authored-by: Miles Obare <[email protected]>
This was referenced May 30, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Geyser plugin receivers can't determine the order of signatures in a block; they receive signatures out of order because geyser streaming happens as transactions are processed in replay, which is done in parallel and with tx randomization.
Summary of Changes
Generate a list of transaction indexes in each slot, and include each index in a new
ReplicaTransactionInfoVersionvariant.This will also enable reworking the Blockstore AddressSignatures column to include the transaction-index in the key, and fix the various getSignatureByAddress ordering/overlapping problems, pending follow-up
To do: