Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor unapplied transaction queue #295

Merged
merged 19 commits into from
May 26, 2022
Merged

Refactor unapplied transaction queue #295

merged 19 commits into from
May 26, 2022

Conversation

heifner
Copy link
Member

@heifner heifner commented May 21, 2022

  • Refactor incoming transaction handling to use unapplied_transaction_queue.
  • Incoming transactions are now stored in the same queue as aborted, forked, and persistent transactions. This allows all expired and applied transactions to be handled together.
  • unapplied_transaction_queue used to have a process_mode where it would not store aborted or forked-out transactions when it knew that the node would never produce a block. This was an optimization because there seemed to be no reason to cache aborted or forked-out transactions. Since the trx has been validated locally and will not be validated again, why keep it around? If you are producing you need to keep it so on a fork-switch the trxs are not lost. But there seemed to be no reason to keep them for speculative mode. However, the unapplied_transaction_queue is also used when validating a block to look up already validated and constructed transaction_metadata. Keeping the already validated trxs in the queue allows for faster validation of received blocks for all the transactions already received by the node.
  • Before this change, when a speculative node's speculative block ran out of cpu or net, it would stop validating transactions until it received a block -- which would cause it to start a new speculative block after processing the received block. With this PR, a new speculative block is created anytime a block is exhausted during speculative transaction processing.
  • Before this change speculative 12th blocks would use the last-block-time-offset-us which doesn't really make any sense since the block is only speculative and will not be broadcast on the network. With this PR, all speculative blocks use the produce-time-offset-us including the 12th block.

Resolves: #205

heifner and others added 18 commits May 21, 2022 11:37
Reorder return_failure_trace flag in structure to match method order
…ilure_trace in unapplied trx queue. Add producer plugin test for trxs
@heifner heifner marked this pull request as ready for review May 21, 2022 17:37
@heifner heifner added the OCI OCI working this issue... label May 21, 2022
{ trx, expiry, persist_until_expired ? trx_enum_type::incoming_persisted : trx_enum_type::incoming, return_failure_trace, std::move( next ) } );
if( insert_itr.second ) added( insert_itr.first );
} else {
if( !(itr->trx_meta == trx) && next ) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure that this logic makes sense, and the comment on the next line seems wrong. I think at the very least we should return whether or not next is defined here (and will we ever not have next set??).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

…as to keep it around in case it was aborted, but it would be removed as a duplicate on next speculative block anyway, so it is meaningless to keep it.

Simplify add_incoming. add_incoming should never change persist_until_expired or return_failure_trace flags, so just report duplicate. No need to update anything.
@heifner heifner requested a review from brianjohnson5972 May 26, 2022 01:43
@heifner heifner merged commit 16a39ce into main May 26, 2022
@heifner heifner deleted the unapplied-trx branch May 26, 2022 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCI OCI working this issue...
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Backport Refactor incoming trx handling
4 participants