Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1. fix(state): Stop reading redundant blocks for every FindHashes and FindHeaders request #4825

Merged
merged 1 commit into from
Jul 26, 2022

Conversation

teor2345
Copy link
Contributor

Motivation

Zebra reads 160-500 redundant blocks for each FindHashes or FindHeaders request.
The FindHeaders requests regularly take 5-10 seconds to answer.

This is a security issue because remote nodes can send these requests.

Solution

  • Find a range of heights, rather than a list of hashes
  • Stop reading redundant blocks and calculating redundant hashes
  • Remove redundant state queries
  • Allow some inconsistencies that can happen during concurrent writes, but don't do concurrent queries yet
  • Simplify some of the code

This PR contains a regression - the header queries used to be partially concurrent, now they block the rest of the state.
The next PR will implement full concurrency.

Review

This is an urgent PR to fix Zebra sync performance.

Reviewer Checklist

  • Code implements Specs and Designs
  • Tests for Expected Behaviour
  • Tests for Errors

@teor2345 teor2345 added C-bug Category: This is a bug P-High 🔥 C-security Category: Security issues I-slow Problems with performance or responsiveness I-integration-fail Continuous integration fails, including build and test failures I-remote-node-overload Zebra can overload other nodes on the network A-state Area: State / database changes labels Jul 26, 2022
@teor2345 teor2345 requested a review from a team as a code owner July 26, 2022 00:54
@teor2345 teor2345 self-assigned this Jul 26, 2022
@teor2345 teor2345 requested review from upbqdn and removed request for a team July 26, 2022 00:54
@teor2345 teor2345 changed the title fix(state): Stop reading redundant blocks for every FindHashes and FindHeaders request 1. fix(state): Stop reading redundant blocks for every FindHashes and FindHeaders request Jul 26, 2022
@codecov
Copy link

codecov bot commented Jul 26, 2022

Codecov Report

Merging #4825 (e772314) into main (ed553a9) will decrease coverage by 0.06%.
The diff coverage is 70.80%.

@@            Coverage Diff             @@
##             main    #4825      +/-   ##
==========================================
- Coverage   78.88%   78.82%   -0.07%     
==========================================
  Files         305      305              
  Lines       38559    38627      +68     
==========================================
+ Hits        30418    30447      +29     
- Misses       8141     8180      +39     

Copy link
Contributor

@oxarbitrage oxarbitrage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@teor2345
Copy link
Contributor Author

Thanks for the review!

All the tests passed except for lightwalletd, which is fixed by PR #4828.
So I'm going to admin-merge this.

@teor2345 teor2345 merged commit 15a55ee into main Jul 26, 2022
@teor2345 teor2345 deleted the slow-block-headers branch July 26, 2022 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-state Area: State / database changes C-bug Category: This is a bug C-security Category: Security issues I-integration-fail Continuous integration fails, including build and test failures I-remote-node-overload Zebra can overload other nodes on the network I-slow Problems with performance or responsiveness
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants