Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sync): prevent synchronizer loop when very close to tip #3854

Merged
merged 3 commits into from
Mar 18, 2022

Conversation

jvff
Copy link
Contributor

@jvff jvff commented Mar 12, 2022

Motivation

Zebra currently has some performance issues when block synchronization is close to the chain tip. During the investigation of the issue, I ran into two causes:

  1. The synchronizer cancels all active block downloads when it exhausts its prospective tips set. This results in the final few block downloads being constantly started and cancelled.
  2. Timeouts when waiting for inputs from the state service during the validation of transactions.

This PR fixes the first cause, and the second cause will very likely be fixed by the refactor to split the state service to increase throughput of read only requests (#3866).

Solution

Only cancel active block download tasks in case there's an error, preventing them from being cancelled when the prospective tips set is exhausted.

Closes #3375

Review

Reviewer Checklist

  • Code implements Specs and Designs
  • Tests for Expected Behaviour
  • Tests for Errors

Follow Up Work

I'd like to have a regression test for this, but I'll try to open a separate optional PR for it.

jvff added 3 commits March 10, 2022 18:34
Replace the use of loop labels and `continue` for control flow, and use
early return from a separate method instead. This also allows removing
the `started_once` flag.
Reduce duplicate code and make the main synchronization methods a little
more concise to improve readability.
Leave active downloads running if the tips have been exhausted, because
it could have reached the chain tip.
@zfnd-bot zfnd-bot bot assigned jvff Mar 12, 2022
@jvff jvff changed the title Fix synchronizer loop when very close to tip fix(sync): prevent synchronizer loop when very close to tip Mar 12, 2022
@codecov
Copy link

codecov bot commented Mar 12, 2022

Codecov Report

Merging #3854 (6b94ca9) into main (7283b4b) will decrease coverage by 0.20%.
The diff coverage is 46.26%.

@@            Coverage Diff             @@
##             main    #3854      +/-   ##
==========================================
- Coverage   78.90%   78.69%   -0.21%     
==========================================
  Files         292      295       +3     
  Lines       33508    33769     +261     
==========================================
+ Hits        26438    26576     +138     
- Misses       7070     7193     +123     

@dconnolly
Copy link
Contributor

This fixes the issue for me 👍

Copy link
Contributor

@teor2345 teor2345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!

Thanks for splitting this PR up into commits, it was very helpful during the review.

zebrad/src/components/sync.rs Show resolved Hide resolved
@conradoplg
Copy link
Collaborator

Is there something preventing this from moving out of draft?

@dconnolly
Copy link
Contributor

@jvff I've been running this since Friday and it's awesome, shall we promote this out of draft?

@jvff jvff added C-bug Category: This is a bug I-slow Problems with performance or responsiveness A-network Area: Network protocol updates or fixes P-Medium ⚡ labels Mar 17, 2022
@jvff jvff marked this pull request as ready for review March 17, 2022 22:28
@jvff jvff requested a review from a team as a code owner March 17, 2022 22:28
@jvff jvff requested review from oxarbitrage and removed request for a team March 17, 2022 22:28
mergify bot added a commit that referenced this pull request Mar 17, 2022
@mergify mergify bot merged commit 78080d8 into main Mar 18, 2022
@mergify mergify bot deleted the fix-sync-loop-when-close-to-tip branch March 18, 2022 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-network Area: Network protocol updates or fixes C-bug Category: This is a bug I-slow Problems with performance or responsiveness
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix slowness when syncing near the tip
4 participants