Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zcash_client_sqlite: Fix errors in progress computation. #1564

Merged

Conversation

nuttycom
Copy link
Contributor

This ensures that we use recover_until_height as an exclusive end height, and makes sure that we are estimating tree size based upon consistent heights between the recovery and scan ranges.

In scan progress computation, `recover_until_height` was incorrectly
being treated as an inclusive end, meaning that it was possible for a
block's notes to be counted both within the recovery range and within
the scanning range.
Copy link

codecov bot commented Oct 10, 2024

Codecov Report

Attention: Patch coverage is 59.88701% with 71 lines in your changes missing coverage. Please review.

Project coverage is 61.65%. Comparing base (e57dc41) to head (279479c).
Report is 12 commits behind head on hotfix/zcash_client_sqlite-0.12.x.

Files with missing lines Patch % Lines
zcash_client_sqlite/src/wallet.rs 58.95% 71 Missing ⚠️
Additional details and impacted files
@@                          Coverage Diff                          @@
##           hotfix/zcash_client_sqlite-0.12.x    #1564      +/-   ##
=====================================================================
+ Coverage                              61.63%   61.65%   +0.01%     
=====================================================================
  Files                                    148      148              
  Lines                                  18821    18832      +11     
=====================================================================
+ Hits                                   11601    11610       +9     
- Misses                                  7220     7222       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

// If none of the wallet's accounts have a recover-until height, then there
// is no recovery phase for the wallet, and therefore the denominator in the
// resulting ratio (the number of notes in the recovery range) is zero.
.unwrap_or_else(|| Some(Ratio::new(0, 0)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine for this API, but is an incompatible change for the SDK-level API. It's fine if we intend to explicitly handle 0:0 at the wallet level now; otherwise we should map that to 1:1 to emulate the old API.

Copy link
Contributor

@str4d str4d Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can cut an 0.13.0 as a hotfix if a breaking change is required. But I'm also inclined to not make this breaking change, unless there's a bug-related reason for it.

Copy link
Contributor

@daira daira Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the mapping to the existing API used by the SDK is done in zcash-light-client-ffi. That's where it makes most sense to do the mapping from 0:0 to 1:1, after adding the numerators and denominators, IMHO (i.e. in Electric-Coin-Company/zcash-light-client-ffi#156).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scan progress denominator is guaranteed to always be nonzero, so adding the numerators and denominators always results in a valid fractional value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In pairing with @str4d, we determined that there may have already been uncommon but possible corner cases where the recovery ratio could have had a zero denominator: this would occur in circumstances where no notes for a given shielded protocol exist in blocks between the wallet's birthday height and the wallet recovery height. This is actually a relatively easily observable situation on testnet, since most of the time there's no shielded testnet activity. As such, 0:0 is not an API-breaking change; we just hadn't previously thought about or documented it adequately.

Comment on lines 922 to 1150
let mut get_tree_size_near = |as_of: BlockHeight| {
stmt_start_tree_size
.query_row(
named_params![":start_height": u32::from(birthday_height)],
|row| row.get::<_, Option<u64>>(0),
)
.optional()
.map(|opt| opt.flatten())
.transpose()
.or_else(|| {
conn.query_row(
&format!(
"SELECT MIN(shard_index)
FROM {table_prefix}_tree_shards
WHERE subtree_end_height >= :start_height
OR subtree_end_height IS NULL",
),
named_params! {
":start_height": u32::from(as_of),
},
|row| {
let min_tree_size = row
.get::<_, Option<u64>>(0)?
.map(|min_idx| min_idx << shard_height);
Ok(min_tree_size)
},
)
.optional()
.map(|opt| opt.flatten())
.transpose()
})
.transpose()
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the same comment to @nuttycom in mob today, that for this kind of highly-nested situation I find imperative code with early returns (?) clearer than functional code with type transposing (despite my general tendency to implement the latter, because in many cases it is great).

Copy link
Contributor

@str4d str4d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed a56de09. Changes to public API need to be discussed (as to whether we keep them, document them in the changelog, and release the hotfix as 0.13.0, or undo them and release the hotfix as 0.12.1).

In either case, this PR is missing a changelog entry documenting the fix.

zcash_client_sqlite/src/wallet.rs Show resolved Hide resolved
zcash_client_sqlite/src/wallet.rs Show resolved Hide resolved
zcash_client_sqlite/src/wallet.rs Show resolved Hide resolved
zcash_client_sqlite/src/wallet.rs Show resolved Hide resolved
Copy link
Contributor

@daira daira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK with comments.

Comment on lines 1148 to 1153
// Get the starting note commitment tree size from the wallet birthday, or failing that
// from the blocks table.
Copy link
Contributor

@daira daira Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"or failing that from the blocks table" seems wrong now; get_tree_size_near doesn't use the blocks table.

Copy link
Contributor

@daira daira Oct 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was mistaken; stmt_start_tree_size does use the blocks table. size_from_subtree_roots doesn't though, so this comment is still not quite accurate (i.e. there is a case in which the tree size comes from another source than the blocks table).

Copy link
Contributor

@str4d str4d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 108e953. My remaining comment feedback is non-blocking.

zcash_client_sqlite/src/wallet.rs Show resolved Hide resolved
@nuttycom
Copy link
Contributor Author

force-pushed to address comments from code review and prepare for the hotfix release.

Copy link
Contributor

@str4d str4d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 279479c

@nuttycom nuttycom merged commit 1f711e0 into zcash:hotfix/zcash_client_sqlite-0.12.x Oct 11, 2024
26 of 27 checks passed
@nuttycom nuttycom deleted the hotfix/scan_progress branch October 11, 2024 14:49
Comment on lines +537 to +538
/// not be treated as authoritative note counts. The denominator of this ratio is
/// guaranteed to be nonzero.
Copy link
Contributor

@daira daira Oct 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this correct?

Copy link
Contributor

@daira daira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post-hoc ACK 279479c. There is a minor inaccuracy in a comment, I think (https://github.com/zcash/librustzcash/pull/1564/files#r1797495621).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants