Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch math & variable-time multiscalar multiplication for redpallas #2288

Merged
merged 12 commits into from
Jun 30, 2021

Conversation

dconnolly
Copy link
Contributor

@dconnolly dconnolly commented Jun 12, 2021

Motivation

We need to validate RedPallas signatures for Orchard/NU5, which check SpendAuth signatures and Binding signatures. For RedJubjub, we utilized a variable-time multiscalar multiplication implementation to do batch validation of those signatures, managed by an async Tower service. This is a useful and performant pattern that we'd like to continue using for both performance reasons but also to maintain our current async work management model.

Solution

Port the RedJubjub multiscalar multiplication and lookup tables to work over pallas::Point, a projective coordinate representation of the short Weierstrass form of the Pallas curve, instead of the Niels representation that was used in computation for the Edwards curve, Jubjub.

I originally thought this multiscalar multiplication would require different math because of the different curve models, but the math is basically the same, it just can use different representations of the points being computed over internally, which are not required to be Niels, whichever works or is fastest for your curve model. Turns out, just using the default pallas::Point, which is projective (vs pallas::Affine), is plenty fast for us, so we can just use that.

Finish the batch API, add correctness tests, and add benchmarks to compare the speed of batched vs unbatched verification (smaller times are better):

image

Resolves #2098

Review

Reviewer Checklist

@conradoplg and @upbqdn especially, would be great to get your eyes on this math, for both knowledge-sharing and in case you know any tips or tricks to make this better. :)

  • Tests for Expected Behaviour
  • Tests for Errors

Follow Up Work

Use this to finish the RedPallasVerifier service: #2287

We currently do not run our benchmarks as part of CI, the zebra-chain::block benchmarks are currently broken by default because os a misconfig of what test resources are available under which feature flags. We may want to run benchmarks as part of CI in a manner similar to our coverage jobs (informational but not blocking) to track these over time and make sure they don't break silently: #2320

@dconnolly dconnolly added A-consensus Area: Consensus rule updates A-rust Area: Updates to Rust code NU-5 Network Upgrade: NU5 specific tasks labels Jun 12, 2021
@dconnolly dconnolly added this to the 2021 Sprint 11 - Zcon2 milestone Jun 12, 2021
@dconnolly dconnolly self-assigned this Jun 12, 2021
@dconnolly dconnolly marked this pull request as ready for review June 12, 2021 19:29
@teor2345
Copy link
Contributor

We currently do not run our benchmarks as part of CI, the zebra-chain::block benchmarks are currently broken by default because os a misconfig of what test resources are available under which feature flags. We may want to run benchmarks as part of CI in a manner similar to our coverage jobs (informational but not blocking) to track these over time and make sure they don't break silently.

@dconnolly can you open a ticket for this follow-up work?

Copy link
Contributor

@teor2345 teor2345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, but I think there's a bug in the batch verification.

It might also help to add some docs to some private functions and types, but I'll leave the final call on that to @conradoplg and @upbqdn.

zebra-chain/benches/redpallas.rs Show resolved Hide resolved
zebra-chain/benches/redpallas.rs Show resolved Hide resolved
fn sigs_with_distinct_keys() -> impl Iterator<Item = Item> {
std::iter::repeat_with(|| {
let mut rng = thread_rng();
let msg = b"";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, we're currently benchmarking failing signatures, because this message does not match the verification message.

So let's:

  • create a constant for the messages, so they all match
  • check for this bug in our other benchmarks
  • consider adding an assert to make sure the signatures are valid (and to avoid optimisations that remove unused code)

We should also re-do the benchmarks, because they're variable time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 5cd8c6f, and retested:

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also asserting that all these signature verifications succeed as part of these benchmarks: 83c4979

zebra-chain/src/primitives/redpallas/batch.rs Outdated Show resolved Hide resolved
zebra-chain/src/primitives/redpallas/batch.rs Outdated Show resolved Hide resolved
zebra-chain/src/primitives/redpallas/batch.rs Outdated Show resolved Hide resolved
zebra-chain/src/primitives/redpallas/batch.rs Outdated Show resolved Hide resolved
str4d
str4d previously approved these changes Jun 14, 2021
Copy link
Contributor

@str4d str4d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not check that the main batch validation logic was correctly ported from redjubjub.

@daira
Copy link
Contributor

daira commented Jun 14, 2021

Couple of typos in the PR description: "muliscalar" -> "multiscalar", "Weierßrass" -> "Weierstraß" or "Weierstrass" (depending on whether you use the orthography from his lifetime or modern German orthography).

@mpguerra mpguerra removed this from the 2021 Sprint 11 - Zcon2 milestone Jun 14, 2021
Copy link
Collaborator

@conradoplg conradoplg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was an interesting read (was a bit nerdsniped too 😆 ). The code is very readable 💯
It seems things haven't changed that much from when I worked with this stuff (wNAFs etc).

Left some comments but nothing major.

zebra-chain/src/primitives/redpallas/batch.rs Show resolved Hide resolved
zebra-chain/src/primitives/redpallas/batch.rs Outdated Show resolved Hide resolved
zebra-chain/src/primitives/redpallas/scalar_mul.rs Outdated Show resolved Hide resolved
@dconnolly dconnolly force-pushed the redpallas-vartime-multiscalar branch from 2ef1525 to ca4de96 Compare June 16, 2021 00:33
@dconnolly
Copy link
Contributor Author

We currently do not run our benchmarks as part of CI, the zebra-chain::block benchmarks are currently broken by default because os a misconfig of what test resources are available under which feature flags. We may want to run benchmarks as part of CI in a manner similar to our coverage jobs (informational but not blocking) to track these over time and make sure they don't break silently.

@dconnolly can you open a ticket for this follow-up work?

Done: #2320

@dconnolly dconnolly requested review from conradoplg and teor2345 June 30, 2021 20:25
@dconnolly dconnolly added this to the 2021 Sprint 12 milestone Jun 30, 2021
@dconnolly dconnolly merged commit ffdaf07 into main Jun 30, 2021
@dconnolly dconnolly deleted the redpallas-vartime-multiscalar branch June 30, 2021 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-consensus Area: Consensus rule updates A-rust Area: Updates to Rust code NU-5 Network Upgrade: NU5 specific tasks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Port/implement batch math and multiscalar multiplication for redpallas
7 participants