feat: Working UltraPlonk for Noir (AztecProtocol/barretenberg#299)

* Make dsl composer agnostic. * change SYSTEM_COMPOSER under stdlib::types to ultra composer type * use ultra logic constraints * in process of debugging, move to using ultra logic constraints * add get_total_circuit_size method * acir format tests showing failures with range constraints of different bit sizes * remove unnecessary comment * (fix) Temporarily add a redundant add-gate for variables that need range constraint < 8 bits. * rename functions * Implement get_solidity_verifier function * Fix no longer available properties * remove constraint system * logic gate changes using plookup * logic gate debugging * test for logic gates passing * last debug things XOR and AND returnign correct results, XOR still failing * cleanup * pedersen_plookup * plookup funcs * add to header * fixed error in pedersen hash when RHS is a circuit constant * added ACIR test for XOR gate pedersen hash test now checks y coordinate * temp disable wasm-opt * Making everything compile with any composer & add a cmake flag to switch on turbo * enable wasm-opt for asyncify but disable optimizations * remove using in header * fixed work queue bug with wasm wasm code path was not correctly storing fft outputs in proving key * added bitwise logic operations into stdlib stdlib method is utility method to provide Composer-agnostic interface due to the use of plookup tables if enabled * updated acir_format to use new stdlib logic class Updated ReadMe to include wasm example that supports gtest filtering * reenable tests * linting fixes * disable binaryen with comment * write instead of read * remove random * WIP * cleanup the debug logging * restore the randomness * only add a zero/one test instead of replacing * remove unused change * changes to make solgen work correctly in bindings * fix join_split_tests.test_deposit_construct_proof * working serialized proving key size and circuit change test for ultra (AztecProtocol/barretenberg#307) * USE_TURBO for join_split * Empty-Commit * Don't default one function; tweak comments. * Empty-Commit --------- Co-authored-by: Suyash Bagad <[email protected]> Co-authored-by: vezenovm <[email protected]> Co-authored-by: Maxim Vezenov <[email protected]> Co-authored-by: zac-williamson <[email protected]> Co-authored-by: kevaundray <[email protected]> Co-authored-by: codygunton <[email protected]>
AztecProtocol · Apr 4, 2023 · 5512fd0 · 5512fd0
1 parent a6648c9
commit 5512fd0
Show file tree

Hide file tree

Showing 56 changed files with 934 additions and 268 deletions.
diff --git a/barretenberg/README.md b/barretenberg/README.md
@@ -132,6 +132,12 @@ cmake --build --preset wasm --target ecc_tests
 wasmtime --dir=.. ./bin/ecc_tests
 ```
 
+To add gtest filter parameters in a wasm context:
+
+```
+wasmtime --dir=.. ./bin/ecc_tests run --gtest_filter=filtertext
+```
+
 ### Fuzzing build
 
 For detailed instructions look in cpp/docs/Fuzzing.md
@@ -160,11 +166,13 @@ cmake --build --preset coverage
 ```
 
 Then run tests (on the mainframe always use taskset and nice to limit your influence on the server. Profiling instrumentation is very heavy):
+
 ```
 taskset 0xffffff nice -n10 make test
 ```
 
 And generate report:
+
 ```
 make create_full_coverage_report
 ```
@@ -174,4 +182,5 @@ The report will land in the build directory in the all_test_coverage_report dire
 Alternatively you can build separate test binaries, e.g. honk_tests or numeric_tests and run **make test** just for them or even just for a single test. Then the report will just show coverage for those binaries.
 
 ### VS Code configuration
+
 A default configuration for VS Code is provided by the file [`barretenberg.code-workspace`](barretenberg.code-workspace). These settings can be overridden by placing configuration files in `.vscode/`.
diff --git a/barretenberg/cpp/CMakeLists.txt b/barretenberg/cpp/CMakeLists.txt
@@ -20,6 +20,14 @@ option(DISABLE_TBB "Intel Thread Building Blocks" ON)
 option(COVERAGE "Enable collecting coverage from tests" OFF)
 option(ENABLE_HEAVY_TESTS "Enable heavy tests when collecting coverage" OFF)
 option(INSTALL_BARRETENBERG "Enable installation of barretenberg. (Projects embedding barretenberg may want to turn this OFF.)" ON)
+option(USE_TURBO "Enable the use of TurboPlonk in barretenberg." OFF)
+
+if(USE_TURBO)
+    message(STATUS "Building barretenberg for TurboPlonk Composer.")
+    add_definitions(-DUSE_TURBO)
+else()
+    message(STATUS "Building barretenberg for UltraPlonk Composer.")
+endif()
 
 if(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64" OR CMAKE_SYSTEM_PROCESSOR MATCHES "arm64")
     message(STATUS "Compiling for ARM.")

diff --git a/barretenberg/cpp/src/CMakeLists.txt b/barretenberg/cpp/src/CMakeLists.txt
@@ -91,7 +91,7 @@ if(WASM)
         $<TARGET_OBJECTS:stdlib_aes128_objects>
         $<TARGET_OBJECTS:stdlib_merkle_tree_objects>
         $<TARGET_OBJECTS:acir_format_objects>
-        $<TARGET_OBJECTS:turbo_proofs_objects>
+        $<TARGET_OBJECTS:acir_proofs_objects>
     )
 
     # With binaryen installed, it seems its wasm backend optimiser gets invoked automatically.
@@ -105,18 +105,24 @@ if(WASM)
         -nostartfiles -O2 -Wl,--no-entry -Wl,--export-dynamic -Wl,--import-memory -Wl,--allow-undefined -Wl,--stack-first -Wl,-z,stack-size=1048576
     )
 
-    find_program(WASM_OPT wasm-opt)
+    # TODO(blaine): Figure out how to Asyncify the wasm output.
+    # Binaryen's Asyncify transform produces wasm that has too many local variables to run in a WebAssembly
+    # instance. This likely would be "solved" by enabling the optimizations to reduce the number of locals,
+    # but using any optimization level results in a wasm file that takes an unusable amount of time to solve the
+    # most simple prood.
 
-    if(NOT WASM_OPT)
-        message(FATAL_ERROR "wasm-opt executable not found. Please install binaryen.")
-    endif()
+    # find_program(WASM_OPT wasm-opt)
 
-    add_custom_command(
-        TARGET barretenberg.wasm
-        POST_BUILD
-        COMMAND wasm-opt "$<TARGET_FILE:barretenberg.wasm>" -O2 --asyncify -o "$<TARGET_FILE:barretenberg.wasm>"
-        VERBATIM
-    )
+    # if(NOT WASM_OPT)
+    #     message(FATAL_ERROR "wasm-opt executable not found. Please install binaryen.")
+    # endif()
+
+    # add_custom_command(
+    #     TARGET barretenberg.wasm
+    #     POST_BUILD
+    #     COMMAND wasm-opt "$<TARGET_FILE:barretenberg.wasm>" -O0 --asyncify -o "$<TARGET_FILE:barretenberg.wasm>"
+    #     VERBATIM
+    # )
 
     if(INSTALL_BARRETENBERG)
         install(TARGETS barretenberg.wasm DESTINATION ${CMAKE_INSTALL_BINDIR})
@@ -151,7 +157,7 @@ if(WASM)
         $<TARGET_OBJECTS:stdlib_aes128_objects>
         $<TARGET_OBJECTS:stdlib_merkle_tree_objects>
         $<TARGET_OBJECTS:acir_format_objects>
-        $<TARGET_OBJECTS:turbo_proofs_objects>
+        $<TARGET_OBJECTS:acir_proofs_objects>
     )
 else()
     # For use when compiling dependent cpp projects
@@ -183,7 +189,7 @@ else()
         $<TARGET_OBJECTS:stdlib_aes128_objects>
         $<TARGET_OBJECTS:stdlib_merkle_tree_objects>
         $<TARGET_OBJECTS:acir_format_objects>
-        $<TARGET_OBJECTS:turbo_proofs_objects>
+        $<TARGET_OBJECTS:acir_proofs_objects>
         $<TARGET_OBJECTS:env_objects>
     )
 

diff --git a/barretenberg/cpp/src/barretenberg/crypto/pedersen/c_bind.cpp b/barretenberg/cpp/src/barretenberg/crypto/pedersen/c_bind.cpp
@@ -1,5 +1,6 @@
 #include "c_bind.hpp"
 #include "pedersen.hpp"
+#include "pedersen_lookup.hpp"
 #include "barretenberg/common/serialize.hpp"
 #include "barretenberg/common/timer.hpp"
 #include "barretenberg/common/mem.hpp"
@@ -21,6 +22,14 @@ WASM_EXPORT void pedersen__compress_fields(uint8_t const* left, uint8_t const* r
     barretenberg::fr::serialize_to_buffer(r, result);
 }
 
+WASM_EXPORT void pedersen_plookup_compress_fields(uint8_t const* left, uint8_t const* right, uint8_t* result)
+{
+    auto lhs = barretenberg::fr::serialize_from_buffer(left);
+    auto rhs = barretenberg::fr::serialize_from_buffer(right);
+    auto r = crypto::pedersen::lookup::compress_native({ lhs, rhs });
+    barretenberg::fr::serialize_to_buffer(r, result);
+}
+
 WASM_EXPORT void pedersen__compress(uint8_t const* inputs_buffer, uint8_t* output)
 {
     std::vector<grumpkin::fq> to_compress;
@@ -29,6 +38,14 @@ WASM_EXPORT void pedersen__compress(uint8_t const* inputs_buffer, uint8_t* outpu
     barretenberg::fr::serialize_to_buffer(r, output);
 }
 
+WASM_EXPORT void pedersen_plookup_compress(uint8_t const* inputs_buffer, uint8_t* output)
+{
+    std::vector<grumpkin::fq> to_compress;
+    read(inputs_buffer, to_compress);
+    auto r = crypto::pedersen::lookup::compress_native(to_compress);
+    barretenberg::fr::serialize_to_buffer(r, output);
+}
+
 WASM_EXPORT void pedersen__compress_with_hash_index(uint8_t const* inputs_buffer, uint8_t* output, uint32_t hash_index)
 {
     std::vector<grumpkin::fq> to_compress;
@@ -46,6 +63,15 @@ WASM_EXPORT void pedersen__commit(uint8_t const* inputs_buffer, uint8_t* output)
     write(output, pedersen_hash);
 }
 
+WASM_EXPORT void pedersen_plookup_commit(uint8_t const* inputs_buffer, uint8_t* output)
+{
+    std::vector<grumpkin::fq> to_compress;
+    read(inputs_buffer, to_compress);
+    grumpkin::g1::affine_element pedersen_hash = crypto::pedersen::lookup::commit_native(to_compress);
+
+    write(output, pedersen_hash);
+}
+
 WASM_EXPORT void pedersen__buffer_to_field(uint8_t const* data, size_t length, uint8_t* r)
 {
     std::vector<uint8_t> to_compress(data, data + length);

diff --git a/barretenberg/cpp/src/barretenberg/crypto/pedersen/c_bind.hpp b/barretenberg/cpp/src/barretenberg/crypto/pedersen/c_bind.hpp
@@ -9,12 +9,15 @@ extern "C" {
 WASM_EXPORT void pedersen__init();
 
 WASM_EXPORT void pedersen__compress_fields(uint8_t const* left, uint8_t const* right, uint8_t* result);
+WASM_EXPORT void pedersen_plookup_compress_fields(uint8_t const* left, uint8_t const* right, uint8_t* result);
 
 WASM_EXPORT void pedersen__compress(uint8_t const* inputs_buffer, uint8_t* output);
+WASM_EXPORT void pedersen_plookup_compress(uint8_t const* inputs_buffer, uint8_t* output);
 
 WASM_EXPORT void pedersen__compress_with_hash_index(uint8_t const* inputs_buffer, uint8_t* output, uint32_t hash_index);
 
 WASM_EXPORT void pedersen__commit(uint8_t const* inputs_buffer, uint8_t* output);
+WASM_EXPORT void pedersen_plookup_commit(uint8_t const* inputs_buffer, uint8_t* output);
 
 WASM_EXPORT void pedersen__buffer_to_field(uint8_t const* data, size_t length, uint8_t* r);
 

diff --git a/barretenberg/cpp/src/barretenberg/dsl/CMakeLists.txt b/barretenberg/cpp/src/barretenberg/dsl/CMakeLists.txt
@@ -1,2 +1,2 @@
 add_subdirectory(acir_format)
-add_subdirectory(turbo_proofs)
+add_subdirectory(acir_proofs)
diff --git a/barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp b/barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp
@@ -1,16 +1,18 @@
 #include "acir_format.hpp"
 
+using namespace plonk::stdlib::types;
+
 namespace acir_format {
 
-void read_witness(TurboComposer& composer, std::vector<barretenberg::fr> witness)
+void read_witness(Composer& composer, std::vector<barretenberg::fr> witness)
 {
     composer.variables[0] = 0;
     for (size_t i = 0; i < witness.size(); ++i) {
         composer.variables[i + 1] = witness[i];
     }
 }
 
-void create_circuit(TurboComposer& composer, const acir_format& constraint_system)
+void create_circuit(Composer& composer, const acir_format& constraint_system)
 {
     if (constraint_system.public_inputs.size() > constraint_system.varnum) {
         std::cout << "too many public inputs!" << std::endl;
@@ -41,7 +43,7 @@ void create_circuit(TurboComposer& composer, const acir_format& constraint_syste
 
     // Add range constraint
     for (const auto& constraint : constraint_system.range_constraints) {
-        composer.decompose_into_base4_accumulators(constraint.witness, constraint.num_bits, "");
+        composer.create_range_constraint(constraint.witness, constraint.num_bits, "");
     }
 
     // Add sha256 constraints
@@ -85,14 +87,14 @@ void create_circuit(TurboComposer& composer, const acir_format& constraint_syste
     }
 }
 
-TurboComposer create_circuit(const acir_format& constraint_system,
-                             std::unique_ptr<bonk::ReferenceStringFactory>&& crs_factory)
+Composer create_circuit(const acir_format& constraint_system,
+                        std::unique_ptr<bonk::ReferenceStringFactory>&& crs_factory)
 {
     if (constraint_system.public_inputs.size() > constraint_system.varnum) {
         std::cout << "too many public inputs!" << std::endl;
     }
 
-    TurboComposer composer(std::move(crs_factory));
+    Composer composer(std::move(crs_factory));
 
     for (size_t i = 1; i < constraint_system.varnum; ++i) {
         // If the index is in the public inputs vector, then we add it as a public input
@@ -119,7 +121,7 @@ TurboComposer create_circuit(const acir_format& constraint_system,
 
     // Add range constraint
     for (const auto& constraint : constraint_system.range_constraints) {
-        composer.decompose_into_base4_accumulators(constraint.witness, constraint.num_bits, "");
+        composer.create_range_constraint(constraint.witness, constraint.num_bits, "");
     }
 
     // Add sha256 constraints
@@ -165,15 +167,15 @@ TurboComposer create_circuit(const acir_format& constraint_system,
     return composer;
 }
 
-TurboComposer create_circuit_with_witness(const acir_format& constraint_system,
-                                          std::vector<fr> witness,
-                                          std::unique_ptr<ReferenceStringFactory>&& crs_factory)
+Composer create_circuit_with_witness(const acir_format& constraint_system,
+                                     std::vector<fr> witness,
+                                     std::unique_ptr<ReferenceStringFactory>&& crs_factory)
 {
     if (constraint_system.public_inputs.size() > constraint_system.varnum) {
         std::cout << "too many public inputs!" << std::endl;
     }
 
-    TurboComposer composer(std::move(crs_factory));
+    Composer composer(std::move(crs_factory));
 
     for (size_t i = 1; i < constraint_system.varnum; ++i) {
         // If the index is in the public inputs vector, then we add it as a public input
@@ -203,7 +205,7 @@ TurboComposer create_circuit_with_witness(const acir_format& constraint_system,
 
     // Add range constraint
     for (const auto& constraint : constraint_system.range_constraints) {
-        composer.decompose_into_base4_accumulators(constraint.witness, constraint.num_bits, "");
+        composer.create_range_constraint(constraint.witness, constraint.num_bits, "");
     }
 
     // Add sha256 constraints
@@ -248,13 +250,13 @@ TurboComposer create_circuit_with_witness(const acir_format& constraint_system,
 
     return composer;
 }
-TurboComposer create_circuit_with_witness(const acir_format& constraint_system, std::vector<fr> witness)
+Composer create_circuit_with_witness(const acir_format& constraint_system, std::vector<fr> witness)
 {
     if (constraint_system.public_inputs.size() > constraint_system.varnum) {
         std::cout << "too many public inputs!" << std::endl;
     }
 
-    auto composer = TurboComposer();
+    auto composer = Composer();
 
     for (size_t i = 1; i < constraint_system.varnum; ++i) {
         // If the index is in the public inputs vector, then we add it as a public input
@@ -284,7 +286,7 @@ TurboComposer create_circuit_with_witness(const acir_format& constraint_system,
 
     // Add range constraint
     for (const auto& constraint : constraint_system.range_constraints) {
-        composer.decompose_into_base4_accumulators(constraint.witness, constraint.num_bits, "");
+        composer.create_range_constraint(constraint.witness, constraint.num_bits, "");
     }
 
     // Add sha256 constraints
@@ -329,7 +331,7 @@ TurboComposer create_circuit_with_witness(const acir_format& constraint_system,
 
     return composer;
 }
-void create_circuit_with_witness(TurboComposer& composer, const acir_format& constraint_system, std::vector<fr> witness)
+void create_circuit_with_witness(Composer& composer, const acir_format& constraint_system, std::vector<fr> witness)
 {
     if (constraint_system.public_inputs.size() > constraint_system.varnum) {
         std::cout << "too many public inputs!" << std::endl;
@@ -363,7 +365,7 @@ void create_circuit_with_witness(TurboComposer& composer, const acir_format& con
 
     // Add range constraint
     for (const auto& constraint : constraint_system.range_constraints) {
-        composer.decompose_into_base4_accumulators(constraint.witness, constraint.num_bits, "");
+        composer.create_range_constraint(constraint.witness, constraint.num_bits, "");
     }
 
     // Add sha256 constraints

diff --git a/barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.hpp b/barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.hpp
@@ -9,6 +9,7 @@
 #include "merkle_membership_constraint.hpp"
 #include "pedersen.hpp"
 #include "hash_to_field.hpp"
+#include "barretenberg/stdlib/types/types.hpp"
 
 namespace acir_format {
 
@@ -35,20 +36,21 @@ struct acir_format {
     friend bool operator==(acir_format const& lhs, acir_format const& rhs) = default;
 };
 
-void read_witness(TurboComposer& composer, std::vector<barretenberg::fr> witness);
+void read_witness(plonk::stdlib::types::Composer& composer, std::vector<barretenberg::fr> witness);
 
-void create_circuit(TurboComposer& composer, const acir_format& constraint_system);
+void create_circuit(plonk::stdlib::types::Composer& composer, const acir_format& constraint_system);
 
-TurboComposer create_circuit(const acir_format& constraint_system,
-                             std::unique_ptr<bonk::ReferenceStringFactory>&& crs_factory);
+plonk::stdlib::types::Composer create_circuit(const acir_format& constraint_system,
+                                              std::unique_ptr<bonk::ReferenceStringFactory>&& crs_factory);
 
-TurboComposer create_circuit_with_witness(const acir_format& constraint_system,
-                                          std::vector<fr> witness,
-                                          std::unique_ptr<ReferenceStringFactory>&& crs_factory);
+plonk::stdlib::types::Composer create_circuit_with_witness(const acir_format& constraint_system,
+                                                           std::vector<fr> witness,
+                                                           std::unique_ptr<ReferenceStringFactory>&& crs_factory);
 
-TurboComposer create_circuit_with_witness(const acir_format& constraint_system, std::vector<fr> witness);
+plonk::stdlib::types::Composer create_circuit_with_witness(const acir_format& constraint_system,
+                                                           std::vector<fr> witness);
 
-void create_circuit_with_witness(TurboComposer& composer,
+void create_circuit_with_witness(plonk::stdlib::types::Composer& composer,
                                  const acir_format& constraint_system,
                                  std::vector<fr> witness);