Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: DSL recursion changes #383

Closed
wants to merge 48 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
69eba2d
Added recursion constraint into dsl + tests
zac-williamson Apr 13, 2023
7a80056
fix dsl ecdsa constraint. Added ecdsa dsl tests
zac-williamson Apr 13, 2023
2436dcf
changed dsl recursion to pass proof/key as witnesses
zac-williamson Apr 16, 2023
292f626
recursive verification key no longer encapsulates a native key
zac-williamson Apr 16, 2023
bc02eff
added serialization test for acir_proofs
zac-williamson Apr 16, 2023
d9656e9
Added serialization methods into dsl for recursive proof composition
zac-williamson Apr 18, 2023
209f813
added verificaiton key hash into RecursionConstraint
zac-williamson Apr 18, 2023
6496228
fixed compiler errors in stdlib_recursion_tests
zac-williamson Apr 18, 2023
5f21e22
feat(dsl)!: Noir recursion updates (#379)
vezenovm Apr 25, 2023
f4253ad
master merge conflicts
vezenovm Apr 25, 2023
25f5574
EnvReferenceString undefined reference to error workaround
vezenovm Apr 25, 2023
1c8cc77
add missing recursion_constraint field from acir_format tests
vezenovm Apr 25, 2023
6109828
add recursion_constraints field back in acir_proofs.test inner circuit
vezenovm Apr 25, 2023
8358741
fix inner circuit in RecursionConstraint.TestRecursionConstraint
vezenovm Apr 25, 2023
b65c442
merge conflictss w/ master
vezenovm Apr 25, 2023
551277b
Empty-Commit
vezenovm Apr 25, 2023
ce46c01
add verify_recursive_proof method for simulating recursive verificati…
vezenovm Apr 28, 2023
87d7b40
master merge conflicts in acir_format and updates to generate method …
vezenovm Apr 28, 2023
11e97be
add ecc module to env package to fix linking of ennv_load_prover_crs …
vezenovm Apr 28, 2023
941b275
add back reference strinng to stdlib recursion key to pass sol verifi…
vezenovm May 1, 2023
6d86e39
remove prints from recursive verifier test
vezenovm May 1, 2023
5a458d6
fix dirty free for when serializing vk to fields, was working on macb…
vezenovm May 3, 2023
7399ea7
Merge branch 'master' into mv/noir-recursion
vezenovm May 5, 2023
07a22f6
fix ecdsa tests after master merge
vezenovm May 5, 2023
74a8327
missing keccak constraints fields in acir format and proofs tests
vezenovm May 5, 2023
18cfb94
one more missing keccak constraint
vezenovm May 5, 2023
7de20cd
merge conflicts w/ master
vezenovm May 15, 2023
155903e
mismatched acir format structs for gcc build
vezenovm May 15, 2023
683a875
missing block constraints in acir tests
vezenovm May 15, 2023
7ecb574
feat(dsl)!: Arbitrary depth recursion (#433)
vezenovm May 16, 2023
a2f6cbb
move export key into dsl package
vezenovm May 22, 2023
b56d3ca
chore: remove unused export key in recursion format from main proof s…
vezenovm May 22, 2023
97ede2e
pr review: moved from_witness comment and renamed from_field_pt_vector
vezenovm May 22, 2023
c64d8b5
moved export transcript in recursion format to DSL package
vezenovm May 22, 2023
a590470
use false for clang-format SortIncludes
vezenovm May 22, 2023
4d262e3
Merge branch 'master' into mv/noir-recursion
vezenovm May 22, 2023
f3baab0
move order of acir functions
vezenovm May 22, 2023
a793b2d
remove ecc bb_module declaration in env package
vezenovm May 22, 2023
f954266
chore: remove usage of magic numbers when slicing g1::affine_element …
vezenovm May 22, 2023
0f0d519
introduce NUM_AGGREGATION_ELEMENTS constant and use it through recurs…
vezenovm May 23, 2023
294c1fa
nit ASSERT(result != -1) in get_public_input_index
vezenovm May 23, 2023
517ed2c
remove unused found var
vezenovm May 23, 2023
92159e0
cast -1
vezenovm May 23, 2023
d8ee388
fix up verify_recursive_proof and add a test for it
vezenovm May 23, 2023
cf9bdd6
moved from tempalte for has_valid_witness_assignments to a flag
vezenovm May 23, 2023
2096676
chore: add comments to AcirProofs.TestVerifyRecursiveProofPass
vezenovm May 23, 2023
bb4404f
Merge branch 'master' into mv/noir-recursion
vezenovm May 24, 2023
5f551ea
minor formatting changes to remove files from PR
zac-williamson May 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions cpp/src/barretenberg/dsl/acir_format/acir_format.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,21 @@ void create_circuit(Composer& composer, const acir_format& constraint_system)
for (const auto& constraint : constraint_system.block_constraints) {
create_block_constraints(composer, constraint);
}

// Add recursion constraints
for (size_t i = 0; i < constraint_system.recursion_constraints.size(); ++i) {
auto& constraint = constraint_system.recursion_constraints[i];
create_recursion_constraints(composer, constraint);

// make sure the verification key records the public input indices of the final recursion output
// (N.B. up to the ACIR description to make sure that the final output aggregation object wires are public
// inputs!)
if (i == constraint_system.recursion_constraints.size() - 1) {
std::vector<uint32_t> proof_output_witness_indices(constraint.output_aggregation_object.begin(),
constraint.output_aggregation_object.end());
composer.set_recursive_proof(proof_output_witness_indices);
}
}
}

Composer create_circuit(const acir_format& constraint_system,
Expand All @@ -116,6 +131,7 @@ Composer create_circuit(const acir_format& constraint_system,
composer.add_variable(0);
}
}

// Add arithmetic gates
for (const auto& constraint : constraint_system.constraints) {
composer.create_poly_gate(constraint);
Expand Down Expand Up @@ -182,6 +198,21 @@ Composer create_circuit(const acir_format& constraint_system,
create_block_constraints(composer, constraint);
}

// Add recursion constraints
for (size_t i = 0; i < constraint_system.recursion_constraints.size(); ++i) {
auto& constraint = constraint_system.recursion_constraints[i];
create_recursion_constraints(composer, constraint);

// make sure the verification key records the public input indices of the final recursion output
// (N.B. up to the ACIR description to make sure that the final output aggregation object wires are public
// inputs!)
if (i == constraint_system.recursion_constraints.size() - 1) {
std::vector<uint32_t> proof_output_witness_indices(constraint.output_aggregation_object.begin(),
Copy link
Contributor

@zac-williamson zac-williamson May 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a method into aggregation_state that unifies these two steps (populating witness indices and calling set_recursive_proof): void aggregation_state::assign_object_to_proof_outputs

Would you mind updating to use it? I think it would allows us to remove the set_recursive_proof method and remove the complexity of having to assign proof_output_witness_indices

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does look like we may still have to assign proof_output_witness_indices as I have to construct an aggregation_state in order to use assign_object_to_proof_outputs. It also looks like constructing the aggregation state will require fetching all the values of the output_aggregation_state from the composer to construct P0 and P1. I can still of course push an update to show how it would look but it does seem that there is some extra complexity to using assign_object_to_proof_outputs even if we can get rid of set_recursive_proof

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that is a good point. One alternative is to pass in a bool add_to_public_inputs parameter into create_recursion_constraint. If add_to_public_inputs == true then we call assign_object_to_proof_outputs on aggregation_state_ct result (line 116)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately that alternative looks to also complicate how we currently construct our constraints. Calling result.assign_object_proof_outputs would mean we are now including the output aggregation object on the result aggregation state directly rather than it being a separate field on our RecursionConstraint. This change would require some larger changes in how we construct our recursion constraints and should be left as a separate optimization.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good. Might be a good idea to create a GH Issue to refactor the AggregationState interface to more easily handle this case + regular recursion w/o exposing internals like proof_output_witness_indices

constraint.output_aggregation_object.end());
composer.set_recursive_proof(proof_output_witness_indices);
}
}

return composer;
}

Expand Down Expand Up @@ -276,6 +307,21 @@ Composer create_circuit_with_witness(const acir_format& constraint_system,
create_block_constraints(composer, constraint);
}

// Add recursion constraints
for (size_t i = 0; i < constraint_system.recursion_constraints.size(); ++i) {
auto& constraint = constraint_system.recursion_constraints[i];
create_recursion_constraints(composer, constraint, true);

// make sure the verification key records the public input indices of the final recursion output
// (N.B. up to the ACIR description to make sure that the final output aggregation object wires are public
// inputs!)
if (i == constraint_system.recursion_constraints.size() - 1) {
std::vector<uint32_t> proof_output_witness_indices(constraint.output_aggregation_object.begin(),
constraint.output_aggregation_object.end());
composer.set_recursive_proof(proof_output_witness_indices);
}
}

return composer;
}
Composer create_circuit_with_witness(const acir_format& constraint_system, std::vector<fr> witness)
Expand Down Expand Up @@ -367,6 +413,21 @@ Composer create_circuit_with_witness(const acir_format& constraint_system, std::
create_block_constraints(composer, constraint);
}

// Add recursion constraints
for (size_t i = 0; i < constraint_system.recursion_constraints.size(); ++i) {
auto& constraint = constraint_system.recursion_constraints[i];
create_recursion_constraints(composer, constraint, true);

// make sure the verification key records the public input indices of the final recursion output
// (N.B. up to the ACIR description to make sure that the final output aggregation object wires are public
// inputs!)
if (i == constraint_system.recursion_constraints.size() - 1) {
std::vector<uint32_t> proof_output_witness_indices(constraint.output_aggregation_object.begin(),
constraint.output_aggregation_object.end());
composer.set_recursive_proof(proof_output_witness_indices);
}
}

return composer;
}
void create_circuit_with_witness(Composer& composer, const acir_format& constraint_system, std::vector<fr> witness)
Expand Down Expand Up @@ -455,6 +516,21 @@ void create_circuit_with_witness(Composer& composer, const acir_format& constrai
for (const auto& constraint : constraint_system.block_constraints) {
create_block_constraints(composer, constraint);
}

// Add recursion constraints
for (size_t i = 0; i < constraint_system.recursion_constraints.size(); ++i) {
auto& constraint = constraint_system.recursion_constraints[i];
create_recursion_constraints(composer, constraint, true);

// make sure the verification key records the public input indices of the final recursion output
// (N.B. up to the ACIR description to make sure that the final output aggregation object wires are public
// inputs!)
if (i == constraint_system.recursion_constraints.size() - 1) {
std::vector<uint32_t> proof_output_witness_indices(constraint.output_aggregation_object.begin(),
constraint.output_aggregation_object.end());
composer.set_recursive_proof(proof_output_witness_indices);
}
}
}

} // namespace acir_format
4 changes: 4 additions & 0 deletions cpp/src/barretenberg/dsl/acir_format/acir_format.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
#include "schnorr_verify.hpp"
#include "ecdsa_secp256k1.hpp"
#include "compute_merkle_root_constraint.hpp"
#include "recursion_constraint.hpp"
#include "block_constraint.hpp"
#include "pedersen.hpp"
#include "hash_to_field.hpp"
Expand All @@ -33,6 +34,7 @@ struct acir_format {
std::vector<PedersenConstraint> pedersen_constraints;
std::vector<ComputeMerkleRootConstraint> compute_merkle_root_constraints;
std::vector<BlockConstraint> block_constraints;
std::vector<RecursionConstraint> recursion_constraints;
// A standard plonk arithmetic constraint, as defined in the poly_triple struct, consists of selector values
// for q_M,q_L,q_R,q_O,q_C and indices of three variables taking the role of left, right and output wire
std::vector<poly_triple> constraints;
Expand Down Expand Up @@ -72,6 +74,7 @@ template <typename B> inline void read(B& buf, acir_format& data)
read(buf, data.pedersen_constraints);
read(buf, data.hash_to_field_constraints);
read(buf, data.fixed_base_scalar_mul_constraints);
read(buf, data.recursion_constraints);
read(buf, data.constraints);
read(buf, data.block_constraints);
}
Expand All @@ -92,6 +95,7 @@ template <typename B> inline void write(B& buf, acir_format const& data)
write(buf, data.pedersen_constraints);
write(buf, data.hash_to_field_constraints);
write(buf, data.fixed_base_scalar_mul_constraints);
write(buf, data.recursion_constraints);
write(buf, data.constraints);
write(buf, data.block_constraints);
}
Expand Down
47 changes: 47 additions & 0 deletions cpp/src/barretenberg/dsl/acir_format/acir_format.test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,50 @@
#include "barretenberg/common/streams.hpp"
#include "barretenberg/serialize/test_helper.hpp"
#include "ecdsa_secp256k1.hpp"

TEST(acir_format, test_a_single_constraint_no_pub_inputs)
{

poly_triple constraint{
.a = 1,
.b = 2,
.c = 3,
.q_m = 0,
.q_l = 1,
.q_r = 1,
.q_o = -1,
.q_c = 0,
};

acir_format::acir_format constraint_system{
.varnum = 4,
.public_inputs = {},
.fixed_base_scalar_mul_constraints = {},
.logic_constraints = {},
.range_constraints = {},
.schnorr_constraints = {},
.ecdsa_constraints = {},
.sha256_constraints = {},
.blake2s_constraints = {},
.keccak_constraints = {},
.hash_to_field_constraints = {},
.pedersen_constraints = {},
.compute_merkle_root_constraints = {},
.block_constraints = {},
.recursion_constraints = {},
.constraints = { constraint },
};

auto composer = acir_format::create_circuit_with_witness(constraint_system, { 0, 0, 1 });

auto prover = composer.create_ultra_with_keccak_prover();
auto proof = prover.construct_proof();

auto verifier = composer.create_ultra_with_keccak_verifier();

EXPECT_EQ(verifier.verify_proof(proof), false);
}

TEST(acir_format, msgpack_logic_constraint)
{
auto [actual, expected] = msgpack_roundtrip(acir_format::LogicConstraint{});
Expand Down Expand Up @@ -97,6 +141,7 @@ TEST(acir_format, test_logic_gate_from_noir_circuit)
.pedersen_constraints = {},
.compute_merkle_root_constraints = {},
.block_constraints = {},
.recursion_constraints = {},
.constraints = { expr_a, expr_b, expr_c, expr_d },
};

Expand Down Expand Up @@ -163,6 +208,7 @@ TEST(acir_format, test_schnorr_verify_pass)
.pedersen_constraints = {},
.compute_merkle_root_constraints = {},
.block_constraints = {},
.recursion_constraints = {},
.constraints = { poly_triple{
.a = schnorr_constraint.result,
.b = schnorr_constraint.result,
Expand Down Expand Up @@ -234,6 +280,7 @@ TEST(acir_format, test_schnorr_verify_small_range)
.pedersen_constraints = {},
.compute_merkle_root_constraints = {},
.block_constraints = {},
.recursion_constraints = {},
.constraints = { poly_triple{
.a = schnorr_constraint.result,
.b = schnorr_constraint.result,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ TEST(up_ram, TestBlockConstraint)
.pedersen_constraints = {},
.compute_merkle_root_constraints = {},
.block_constraints = { block },
.recursion_constraints = {},
.constraints = {},
};

Expand Down
3 changes: 3 additions & 0 deletions cpp/src/barretenberg/dsl/acir_format/ecdsa_secp256k1.test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ TEST(ECDSASecp256k1, TestECDSAConstraintSucceed)
.pedersen_constraints = {},
.compute_merkle_root_constraints = {},
.block_constraints = {},
.recursion_constraints = {},
.constraints = {},
};

Expand Down Expand Up @@ -134,6 +135,7 @@ TEST(ECDSASecp256k1, TestECDSACompilesForVerifier)
.pedersen_constraints = {},
.compute_merkle_root_constraints = {},
.block_constraints = {},
.recursion_constraints = {},
.constraints = {},
};
auto crs_factory = std::make_unique<proof_system::ReferenceStringFactory>();
Expand Down Expand Up @@ -167,6 +169,7 @@ TEST(ECDSASecp256k1, TestECDSAConstraintFail)
.pedersen_constraints = {},
.compute_merkle_root_constraints = {},
.block_constraints = {},
.recursion_constraints = {},
.constraints = {},
};

Expand Down
Loading