feat: prove openings of masking polynomials in ECCVM and Translator #9726

iakovenkos · 2024-11-04T17:15:53Z

As a part of ZK-fication of Honk, we have to mask the evaluations of round univariates that the prover sends to the verifier. The evaluations were masked in Sumcheck in PR #7517. However, the logic for proving evaluations of Libra masking polynomials was missing. This PR fixes this issue and enables efficient batch opening of these polynomials.

Added necessary logic to Shplonk Prover, Shplemini Prover, and Shplemini Verifer
Better handling of the ZKSumcheckData
Removed methods and reverted changes that became obsolete because of the new ZK strategy
Enabled the opening of Libra masking univariates in ECCVM and Translator

iakovenkos · 2024-11-04T17:19:13Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplonk.hpp

    {
        const size_t num_opening_claims = opening_claims.size();

-        // {ẑⱼ(r)}ⱼ , where ẑⱼ(r) = 1/zⱼ(r) = 1/(r - xⱼ)
+        // {ẑⱼ(z)}ⱼ , where ẑⱼ(r) = 1/zⱼ(z) = 1/(z - xⱼ)


iakovenkos · 2024-11-05T11:44:46Z

barretenberg/cpp/src/barretenberg/commitment_schemes/kzg/kzg.test.cpp

@@ -41,6 +41,42 @@ TYPED_TEST(KZGTest, single)
    EXPECT_EQ(this->vk()->pairing_check(pairing_points[0], pairing_points[1]), true);
 }

+/**


We usually commit to and open univariate polynomials represented in the monomial basis. Here is a correct flow for the univariates given in the Lagrange basis.

iakovenkos · 2024-11-05T11:47:20Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplemini.hpp

@@ -439,5 +466,70 @@ template <typename Curve> class ShpleminiVerifier_ {
            commitments.emplace_back(std::move(fold_commitments[j]));
        }
    }
+


main feature of this PR - the method enabling efficient opening of claimed evaluations of Libra masking univariates. no extra challenges or batch mul calls required

iakovenkos · 2024-11-05T11:50:43Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplonk.hpp

@@ -113,6 +134,22 @@ template <typename Curve> class ShplonkProver_ {
            idx++;
        }

+        // Take into account the constant proof size in Gemini


it's just a usual shplonk stuff taking into account that we have const proof size in gemini

iakovenkos · 2024-11-05T11:51:33Z

barretenberg/cpp/src/barretenberg/eccvm/eccvm_flavor.hpp

@@ -68,7 +68,7 @@ class ECCVMFlavor {
    using Relations = Relations_<FF>;
    using LookupRelation = ECCVMLookupRelation<FF>;

-    static constexpr size_t MAX_PARTIAL_RELATION_LENGTH = compute_max_partial_relation_length<Relations, HasZK>();
+    static constexpr size_t MAX_PARTIAL_RELATION_LENGTH = compute_max_partial_relation_length<Relations>();


don't need this anymore as we are switching to a different evaluation masking technique. same in translator

iakovenkos · 2024-11-05T11:52:45Z

barretenberg/cpp/src/barretenberg/sumcheck/zk_sumcheck_data.hpp


+    // Default constructor


added a constructor here, moved a bunch of method from sumcheck.hpp

iakovenkos · 2024-11-05T11:54:45Z

barretenberg/cpp/src/barretenberg/sumcheck/sumcheck.hpp

@@ -190,13 +188,11 @@ template <typename Flavor> class SumcheckProver {
    SumcheckOutput<Flavor> prove(ProverPolynomials& full_polynomials,
                                 const bb::RelationParameters<FF>& relation_parameters,
                                 const RelationSeparator alpha,
-                                 const std::vector<FF>& gate_challenges)
+                                 const std::vector<FF>& gate_challenges,
+                                 ZKSumcheckData<Flavor> zk_sumcheck_data = ZKSumcheckData<Flavor>())


decoupled the creation of a ZKSumcheckData object from the prove method. now it's taken care by a prover that uses a constructor to create these data and then feeds it to the prove method

iakovenkos · 2024-11-05T11:56:19Z

barretenberg/cpp/src/barretenberg/sumcheck/sumcheck_round.hpp

        if constexpr (Flavor::HasZK) {
            output += full_libra_purported_value.value();
+            if constexpr (IsECCVMRecursiveFlavor<Flavor>) {
+                output.self_reduce();


we have to self_reduce here

this is only necessary for the ECCVM? is it clear why?

it's only the ECCVMRecursiveFlavor, because it's operating with the scalars in bigfield

…col/aztec-packages into si/zk-sumcheck-plus-shplemini

ledwards2225

Looks great, just some minor suggestions about style/comments. Also, is there any more cleanup that can be done in the Flavors as a result of this? I'm thinking about the erroneous NUM_ALL_WITNESS_ENTITIES in the MegaFlavor for example..

ledwards2225 · 2024-11-06T15:19:18Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplemini.hpp

-        OpeningClaim batched_claim = ShplonkProver::prove(commitment_key, opening_claims, transcript);
+        // Create opening claims for Libra masking univariates
+        std::vector<OpeningClaim> libra_opening_claims;
+        if (!libra_univariates.empty()) {


its a little redundant to have this check prior to the loop. If it's empty, the loop will just do nothing right?

ledwards2225 · 2024-11-06T15:22:04Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplemini.hpp

@@ -254,6 +271,16 @@ template <typename Curve> class ShpleminiVerifier_ {
        commitments.emplace_back(g1_identity);
        scalars.emplace_back(constant_term_accumulator);

+        if (!libra_univariate_evaluations.empty()) {


Is there a more clarifying condition that could be used here? e.g is this always empty for non-zk flavors and always non-empty for zk flavors?

exactly, but it's handled somewhat manually because the PCS doesn't know anything about the Flavor. added a clarifying comment

this is probably an indication that the PCS now needs to be templated on Flavor instead of just Curve

agree. I think I'll stop adding stuff to Shplemini quite soon and then we'll have to refactor it, because its interface is getting quite messy

ledwards2225 · 2024-11-06T15:26:19Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplemini.hpp

+        size_t num_libra_univariates = libra_univariate_commitments.size();
+
+        // compute Shplonk denominators and invert them
+        for (size_t idx = 0; idx < num_libra_univariates; idx++) {


Is the number of libra univariates different from the size of the multivariate challenge? If not it might be better to use a range loop here on multivariate_challenge

yeah, the size of the challenge is CONST_PROOF_SIZE_LOG_N, and the number of libra univariates if the honest log circuit size. prob we'll need to constify it at some point, but I decided to avoid it here

ledwards2225 · 2024-11-06T15:30:34Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplemini.hpp

+        }
+
+        // needed to keep track of the constant term contribution
+        const size_t idx_of_constant_term = commitments.size() - 1;


another option here is to simply define contant_term or constant_scalar as a ref then update it below. E.g. Fr& constant_scalar = scalars.back(); then constant_scalar += constant_term;. just personal preference I suppose

ledwards2225 · 2024-11-06T15:31:35Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplemini.test.cpp

+    for (size_t idx = 0; idx < log_n; idx++) {
+        // generate random polynomial
+        Polynomial libra_polynomial = Polynomial::random(LIBRA_UNIVARIATE_LENGTH);
+        // create a univariate with the same coefficients (to store an array intead of a vector)


ledwards2225 · 2024-11-06T15:35:18Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplemini.test.cpp

+        Polynomial libra_polynomial = Polynomial::random(LIBRA_UNIVARIATE_LENGTH);
+        // create a univariate with the same coefficients (to store an array intead of a vector)
+        bb::Univariate<Fr, LIBRA_UNIVARIATE_LENGTH> libra_univariate;
+        for (size_t i = 0; i < LIBRA_UNIVARIATE_LENGTH; i++) {


FYI it looks like Univariate has it's own random_element() method - might be cleaner

thanks! it's ugly here: Univariates are in the Lagrange basis, Polynomials are in the monomial, which means we can't commit to a Univariate without computing its coefficients in the monomial basis or computing the Lagrange basis using SRS + I want to store the masking data as a std::vector<Univariate<FF, 12>> (could/will be turned into std::array) instead of std::vector

ledwards2225 · 2024-11-06T15:35:57Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplemini.test.cpp

+    auto eval3_shift = poly3.evaluate_mle(mle_opening_point, true);
+
+    // Collect multilinear evaluations for input to prover
+    // std::vector<Fr> multilinear_evaluations = { eval1, eval2, eval3, eval4, eval2_shift, eval3_shift };


ledwards2225 · 2024-11-06T15:37:48Z

barretenberg/cpp/src/barretenberg/commitment_schemes/shplonk/shplonk.hpp

@@ -61,7 +62,20 @@ template <typename Curve> class ShplonkProver_ {
            Q.add_scaled(tmp, current_nu);
            current_nu *= nu;
        }
+        for (size_t idx = opening_claims.size(); idx < CONST_PROOF_SIZE_LOG_N + 2; idx++) {


would be nice to add a comment explaining the significance of CONST_PROOF_SIZE_LOG_N + 2

added a comment

ledwards2225 · 2024-11-06T15:40:54Z

barretenberg/cpp/src/barretenberg/relations/ecc_vm/ecc_lookup_relation.hpp

@@ -82,15 +82,18 @@ polynomials,
        using View = typename Accumulator::View;

        if constexpr (read_index == 0) {
+


are these spaces intentional?

ledwards2225 · 2024-11-06T15:44:28Z

barretenberg/cpp/src/barretenberg/sumcheck/sumcheck_round.hpp

        if constexpr (Flavor::HasZK) {
            output += full_libra_purported_value.value();
+            if constexpr (IsECCVMRecursiveFlavor<Flavor>) {
+                output.self_reduce();


this is only necessary for the ECCVM? is it clear why?

…9726) As a part of ZK-fication of Honk, we have to mask the evaluations of round univariates that the prover sends to the verifier. The evaluations were masked in Sumcheck in PR #7517. However, the logic for proving evaluations of Libra masking polynomials was missing. This PR fixes this issue and enables efficient batch opening of these polynomials. * Added necessary logic to Shplonk Prover, Shplemini Prover, and Shplemini Verifer * Better handling of the ZKSumcheckData * Removed methods and reverted changes that became obsolete because of the new ZK strategy * Enabled the opening of Libra masking univariates in ECCVM and Translator

iakovenkos added 14 commits October 28, 2024 17:53

shplemini supports opening of libra masking univariates

2c37dc4

clean-up

7bbacc2

de-coupled sumcheck from zk sumcheck data

c78ed6e

eccvm native proves/verifies evals of masking polys

fe93b4d

debugging eccvm recursive

df7f731

debugggging still

0b3e50a

Merge branch 'master' into si/zk-sumcheck-plus-shplemini

525ae12

eccvm recursive fixed

4298d52

opening proofs for libra polynomials in translator

bf468eb

translator relation degree changes reverted

ab9b81e

some clean-up

aca70fa

removed redundant line

b7584ba

clean-up

6403477

Merge branch 'master' into si/zk-sumcheck-plus-shplemini

bec76b9

iakovenkos commented Nov 4, 2024

View reviewed changes

iakovenkos and others added 3 commits November 4, 2024 18:16

fix: link zero constraint relation in translator

8a1e95f

fix bb native tests

2504556

Merge branch 'master' into si/zk-sumcheck-plus-shplemini

d18f0c2

iakovenkos commented Nov 5, 2024

View reviewed changes

iakovenkos added 2 commits November 5, 2024 12:14

clean up

937af2a

Merge branch 'si/zk-sumcheck-plus-shplemini' of github.com:AztecProto…

6824085

…col/aztec-packages into si/zk-sumcheck-plus-shplemini

iakovenkos self-assigned this Nov 5, 2024

fix build

95d4160

iakovenkos requested a review from ledwards2225 November 5, 2024 16:09

ledwards2225 approved these changes Nov 6, 2024

View reviewed changes

clean-up after review

69b913e

iakovenkos added e2e-prover-full CI: Enables this CI job. crypto cryptography labels Nov 6, 2024

Merge branch 'master' into si/zk-sumcheck-plus-shplemini

ed57e42

ludamad added e2e-prover-full CI: Enables this CI job. e2e Enables e2e jobs in BB change PRs. and removed e2e-prover-full CI: Enables this CI job. labels Nov 6, 2024

iakovenkos and others added 2 commits November 7, 2024 01:28

Merge branch 'master' into si/zk-sumcheck-plus-shplemini

1775d41

resolve conflicts with 'master'

e8c691a

iakovenkos removed e2e-prover-full CI: Enables this CI job. e2e Enables e2e jobs in BB change PRs. labels Nov 7, 2024

Merge branch 'master' into si/zk-sumcheck-plus-shplemini

5f62335

iakovenkos merged commit f1cdc2d into master Nov 7, 2024
47 checks passed

iakovenkos deleted the si/zk-sumcheck-plus-shplemini branch November 7, 2024 12:49

AztecBot mentioned this pull request Nov 7, 2024

chore(master): Release 0.63.0 #9651

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: prove openings of masking polynomials in ECCVM and Translator #9726

feat: prove openings of masking polynomials in ECCVM and Translator #9726

iakovenkos commented Nov 4, 2024 •

edited

Loading

iakovenkos Nov 4, 2024

iakovenkos Nov 5, 2024

iakovenkos Nov 5, 2024

iakovenkos Nov 5, 2024

iakovenkos Nov 5, 2024

iakovenkos Nov 5, 2024

iakovenkos Nov 5, 2024

iakovenkos Nov 5, 2024

ledwards2225 Nov 6, 2024

iakovenkos Nov 6, 2024

ledwards2225 left a comment

ledwards2225 Nov 6, 2024

ledwards2225 Nov 6, 2024

iakovenkos Nov 6, 2024

ledwards2225 Nov 6, 2024

iakovenkos Nov 6, 2024

ledwards2225 Nov 6, 2024

iakovenkos Nov 6, 2024

ledwards2225 Nov 6, 2024

ledwards2225 Nov 6, 2024

ledwards2225 Nov 6, 2024

iakovenkos Nov 6, 2024

ledwards2225 Nov 6, 2024

ledwards2225 Nov 6, 2024

iakovenkos Nov 6, 2024

ledwards2225 Nov 6, 2024

ledwards2225 Nov 6, 2024

		@@ -82,15 +82,18 @@ polynomials,
		using View = typename Accumulator::View;

		if constexpr (read_index == 0) {

feat: prove openings of masking polynomials in ECCVM and Translator #9726

feat: prove openings of masking polynomials in ECCVM and Translator #9726

Conversation

iakovenkos commented Nov 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ledwards2225 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iakovenkos commented Nov 4, 2024 •

edited

Loading