Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Non finalized block commit channel #5210

Closed
wants to merge 32 commits into from
Closed

Non finalized block commit channel #5210

wants to merge 32 commits into from

Conversation

arya2
Copy link
Contributor

@arya2 arya2 commented Sep 20, 2022

Motivation

We want to make state block writes concurrent by moving block writes into their own task.

This PR sends write commands with queued non-finalized blocks down a channel to the writer task

Part of #4937.

Designs

Solution

  • Commit finalized blocks to the state in a separate task

Review

Some tests are failing, not yet ready for review

Reviewer Checklist

  • Code implements Specs and Designs
  • Tests for Expected Behaviour
  • Tests for Errors

teor2345 and others added 30 commits September 16, 2022 16:08
@teor2345
Move the finalized block queue into the StateService
aa3cae4
@teor2345
Move the queued_blocks module to the state service
f2809c5
@teor2345
Move QueuedFinalized into queued_blocks
b2a2526
@teor2345
Move the queued_blocks tests into their own module
cb2c0c4
@teor2345
Make the FinalizedState cloneable
8aeebda
@teor2345
Add a new block commit task and channels, that don't do anything yet
79c8173
@teor2345
Add last_block_hash_sent to the state service, to avoid database acce…
340d394 
…sses
@teor2345
Update last_block_hash_sent regardless of commit errors
fb7947c
@teor2345
Rename a field to StateService.max_queued_finalized_height
e0584a3
@teor2345
Commit finalized blocks to the state in a separate task
ca8ddf0
@teor2345
Check for panics in the block write task
ced7ffb
@teor2345
Wait for the block commit task in tests, and check for errors
d885e3c
@teor2345
Always run a proptest that sleeps once
f94cbc6
@teor2345
Add extra debugging to state shutdowns
dcedc8b
@teor2345
Work around a RocksDB shutdown bug
c0c106d
@teor2345
Close the finalized block channel when we're finished with it
8863835
@teor2345
Only reset state queue once per error
1dce756
@teor2345
Update some TODOs
9fd7b85
@teor2345
Add a module doc comment
ec63d4c
@teor2345
Drop channels and check for closed channels in the block commit task
3571ac1
@teor2345
Close state channels and tasks on drop
af15b5d
@teor2345
Remove some duplicate fields across StateService and ReadStateService
4d11b7a
@teor2345
Try tweaking the shutdown steps
23094b8
@teor2345
Update and clarify some comments
633e33e
@teor2345
Clarify another comment
7d9fcfa
@teor2345
Don't try to cancel RocksDB background work on drop
21f902b
@teor2345
Fix up some comments
fbb6fae
@teor2345
Remove some duplicate code
dd8ed5b
@teor2345
Remove redundant workarounds for shutdown issues
09349cb
@arya2
moves commit ops in queue_and_commit_non_finalized method of StateSer…
285baea 
…vice to block commit task
@arya2 arya2 requested review from upbqdn and teor2345 September 20, 2022 16:07
@arya2 arya2 closed this Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@upbqdn upbqdn Awaiting requested review from upbqdn

@teor2345 teor2345 Awaiting requested review from teor2345

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@arya2 @teor2345