Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Put parent_header to PreloadUnverified, and keep small block in memory #4461

Closed
wants to merge 360 commits into from
Closed

Put parent_header to PreloadUnverified, and keep small block in memory #4461

wants to merge 360 commits into from

Conversation

eval-exec
Copy link
Collaborator

@eval-exec eval-exec commented May 15, 2024 •
edited
Loading

LonelyBlockHash

What problem does this PR solve?

  1. keep small block in LonelyBlockHash
    Change
pub struct LonelyBlockHash {
    /// block
    pub block_number_and_hash: BlockNumberAndHash,

to

pub struct LonelyBlockHash {
    /// block
    pub block_number_and_hash: Either<BlockNumberAndHash, Arc<BlockView>>,
  1. put parent_header to PreloadUnverified

What's Changed:

Related changes

  • Keep small block in PreloadUnverified's channel, then we can reduce many get operation from RocksDB.
  • put parent_header to PreloadUnverified, then we may reduce one get operation from RocksDB.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code ci-runs-only: [ quick_checks,linters ]

Side effects

  • None

Release note 

None: Exclude this PR from the release note.

@eval-exec
Remove useless import orphan_block_pool in ckb-sync
f12644f
@eval-exec
Fix blocking process block usage in benches/benches/benchmarks/always…
1f83b3e 
…_success.rs
@eval-exec
Fix blocking process block usage in benches/benches/benchmarks/overal…
6123d60 
…l.rs
@eval-exec
Fix blocking process block usage in benches/benches/benchmarks/resolv…
9d4ae55 
…e.rs
@eval-exec
Fix blocking process block usage in benches/benches/benchmarks/secp_2…
17a571e 
…in2out.rs
@eval-exec
let ckb-import and ckb-replay construct ChainService with SharedPackage
d3420e3
@eval-exec
Fix Unit test: ckb-light-client-protocol-server::tests/utils/chain.rs
3896d3b
@eval-exec
Fix Unit test: ckb-verification::contextual/src/tests/uncle_verifier.rs
0e4e859
@eval-exec
Fix Unit test: ckb-verification::contextual/src/tests/contextual_bloc…
684654c 
…k_verifier.rs
@eval-exec
Fix Unit test: ckb-sync::src/tests/util.rs
50eae6e
@eval-exec
Fix blocking process block usage in sync/src/relayer/tests/helper.rs
89da541
@eval-exec
Fix Unit test of blocking process block usage in ckb-sync
6b9cb22
@eval-exec
Use blocking process_block method for benches/benches/benchmarks/secp…
e6d45af 
…_2in2out.rs
@eval-exec
Benches: Construct ChainService with SharedPackage provided verify_fa…
bae4525 
…iled_block_rx
@eval-exec
Modify BlockStatus by SyncShared.Shared
7b2297b
@eval-exec
Unit Test: Use SharedPackage to construct ChainService
e32648a
@eval-exec
Unit Test: Use SharedPackage to construct ChainService in ckb-chain::…
8019a93 
…find_fork.rs
@eval-exec
Unit test: Modify ChainService initialization to include `pack.take…
c5fa529 
…_verify_failed_block_tx()` parameter
@eval-exec
Unit test: Modify the new function in ChainService to include ver…
4fb9013 
…ify_failed_block_tx parameter
@eval-exec
Unit test: Initialization of ChainService need pack.take_verify_faile…
10eba3a 
…d_block_tx
@eval-exec
Add more minor ticks to sync progress chart
b2d51c5
@eval-exec
Add draft mermaid sequence diagram for develop branch
bd7bad0
@eval-exec
Add draft mermaid sequence diagram for asynchronous block download
1b4256d
@eval-exec
Add unverified block info to sync_state rpc
0c05d58
@eval-exec
Upgrade Synchronizer disconnect log level from debug to info
cd6d91f
@eval-exec
Add tip_hash and tip_number to sync_state rpc
d1d5fec
@eval-exec
Set receive exit signal log from debug to info
2008b2d
@eval-exec
Let HeaderMap stats feature use info log
5d425a2
@eval-exec
Activate HeaderMap stats profiling feature in make profiling
cac179c 
Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Return removed inflight blocks count when disconnect
6a8bb07 
Signed-off-by: Eval EXEC <[email protected]>
eval-exec and others added 24 commits May 13, 2024 10:39
@eval-exec
Fix sync and relayer unit test by verify_callback
a0c6ff8
@eval-exec
Remove VerifyFailedInfo, remove verify_failed_block channel
86deeb3
@eval-exec
Remove broadcast_compact_block channel
565800b
@eval-exec
Add debug log for block_fetcher->get_ancestor
156b86e
@quake @eval-exec
chore: remove channel send error callback
00fae3d
@quake @eval-exec
chore: remove UnverifiedBlock
e066c68
@eval-exec
Find and verify unverified blocks on ckb startup
75a8007
@eval-exec
Fix cargo clippy warnings
e33733f 
Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Use Release to store, Acquire to read for atomic: `is_verifyi…
63f767a 
…ng_unverified_blocks_on_startup`

Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Do not accept descendants if parent is invalid
b098b1f 
Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Update rpc docs, fix CI test
f1ad952 
Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Release ckb-async-download rc0
8f18471
@eval-exec
Do not re-store_unverified_block if it is still verifying by ConsumeU…
fab3055 
…nverified
@eval-exec
Apply cargo clippy suggestions
6e7bbcf
@eval-exec
Rebased develop, regenerate rpc docs, remove orphan pool total_size l…
e86b6c9 
…imitation

Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Prestore block to db before orphan pool, use fill unverified thread t…
c9617c4 
…o pre load unverified block

Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Shared::get_block_status always read BlockExt from snapshot
23386c1 
Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Release ckb-async-download rc1
47e2ff3
@eval-exec
Apply code review suggestions
0f8c8c8 
Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Remove block_status_map if block is is_internal_db_error
e6ca07b
@eval-exec
Delete unverified block if verify failed
f7d6b00
@eval-exec
Add SyncInvalid integration test
a0532cb
@eval-exec
Fix cargo clippy for integration test
95c26d9 
Signed-off-by: Eval EXEC <[email protected]>
@eval-exec
Temporary put parent header to PreloadUnverified, put small BlockView to
ff3cab9 
LonelyBlockHash

Signed-off-by: Eval EXEC <[email protected]>
@eval-exec eval-exec requested a review from a team as a code owner May 15, 2024 00:47
@eval-exec eval-exec requested review from zhangsoledad and removed request for a team May 15, 2024 00:47
@eval-exec eval-exec marked this pull request as draft May 15, 2024 00:50
@eval-exec eval-exec closed this May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhangsoledad zhangsoledad Awaiting requested review from zhangsoledad zhangsoledad is a code owner automatically assigned from nervosnetwork/ckb-dev

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@eval-exec @contrun @quake