Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: all hashes in ts #3333

Merged
merged 13 commits into from
Nov 18, 2023
Merged

chore: all hashes in ts #3333

merged 13 commits into from
Nov 18, 2023

Conversation

charlielye
Copy link
Contributor

@charlielye charlielye commented Nov 16, 2023

This is mostly just removing a bunch of async/await stuff as we convert the final 3 functions in abis.ts to use foundation/crypto/pedersen.

There are two things that are iffy, but not sure if that should block a merge as everything passes:

  • vkHash, I've hardcoded a couple of values I don't fully understand (coset generator and root, which are pulled from evalutation domain in the C++).
  • computePublicCallStackItemHash the C++ did something funky when isExecutionRequest is true. I don't fully understand what and if it needs reproducing here, but again, everything passes.

Have left the commented C++ code in the two places above for reference.

@AztecBot
Copy link
Collaborator

AztecBot commented Nov 17, 2023

Benchmark results

No metrics with a significant change found.

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit 48d8c7fd and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 8 txs 32 txs 128 txs
l1_rollup_calldata_size_in_bytes 45,444 179,588 716,132
l1_rollup_calldata_gas 223,008 868,220 3,449,576
l1_rollup_execution_gas 842,095 3,595,328 22,204,945
l2_block_processing_time_in_ms 2,047 (+2%) 7,629 30,214 (+1%)
note_successful_decrypting_time_in_ms 293 874 (+1%) 3,196 (+1%)
note_trial_decrypting_time_in_ms 95.9 (+3%) 82.0 (+12%) 132 (-2%)
l2_block_building_time_in_ms 20,545 81,618 326,072
l2_block_rollup_simulation_time_in_ms 11,944 47,366 (-1%) 188,397
l2_block_public_tx_process_time_in_ms 8,555 34,119 137,179 (+1%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric 5 blocks 10 blocks
node_history_sync_time_in_ms 21,886 (+1%) 42,796
note_history_successful_decrypting_time_in_ms 2,025 (-1%) 4,077
note_history_trial_decrypting_time_in_ms 121 (+1%) 145 (-18%)
node_database_size_in_bytes 1,629,170 1,099,493
pxe_database_size_in_bytes 29,748 59,307

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit circuit_simulation_time_in_ms circuit_input_size_in_bytes circuit_output_size_in_bytes
private-kernel-init 764 61,697 18,905
private-kernel-ordering 125 24,297 8,153
base-rollup 1,771 656,428 873
root-rollup 168 (-1%) 4,072 1,097
private-kernel-inner 788 (+1%) 81,568 18,905
public-kernel-private-input 568 41,519 18,905
public-kernel-non-first-iteration 406 41,561 18,905
merge-rollup 15.2 2,592 873

Miscellaneous

Transaction sizes based on how many contracts are deployed in the tx.

Metric 0 deployed contracts 1 deployed contracts
tx_size_in_bytes 8,787 27,547

@charlielye charlielye changed the base branch from master to cl/compute_function_tree_root_in_ts November 17, 2023 19:42
@charlielye charlielye requested a review from ludamad November 17, 2023 19:54
* We once again codegen a sync version of bb.js api, and use it instead
of the hand rolled Pedersen class.
* This improves upon how we called into the wasm. Previously when it was
in a worker, every mem copy/slice and call was an individual call over
the boundary. Now the binder is directly in the wasm wrapper, and there
is a separation of "enriched type" construction. i.e. the call over the
boundary only contains primitives, and contains all the necessary info,
and the returned value contains all the sliced memory, then the
codegened api does the enriching.
* The above also means we don't need two different binders/heap
allocator handlers for async/sync apis.
* It should also mean we never worry about two async calls being made
over the boundary at once and corrupting the shared scratch space.
* We construct a singleton of the sync api on demand in bb.js, so the
instance can be used repeatedly.
* At present it's just used in `foundation/crypto/pedersn`, but can now
start leveraging it in Aes/Ecdsa/Grumpkin classes etc.
Base automatically changed from cl/compute_function_tree_root_in_ts to master November 18, 2023 09:59
@charlielye charlielye enabled auto-merge (squash) November 18, 2023 10:05
@charlielye
Copy link
Contributor Author

This PR was meant to be a lot smaller, but it looks like: #3345 got accidentaly merged into it early.

* Branch name perhaps not totally honest, as we're not using the codegen
api. A simpler stepping stone was to substitute in the bb.js wasm for
the circuits wasm in the various classes that used it.
* Switching to use api for making the wasm calls will require updating
the bindings to follow the bindgen calling convention. I've done this
for aes, but can do the others later.
* No more async needed to create aes/schnorr/grumpkin etc.
* We can probably get rid of the classes themselves soon and just have
free functions.

If we seach the codebase now for `await CircuitsWasm.get()` there's only
a handful of lines left, mostly related to what I think are deprecated
circuit simulation calls. Once we get rid of these, we can get rid of
building the circuits in CI.
Copy link

Removed dependencies detected. Learn more about Socket for GitHub ↗︎

🚮 Removed packages: @rushstack/[email protected]

@ludamad
Copy link
Collaborator

ludamad commented Nov 18, 2023

This PR was meant to be a lot smaller, but it looks like: #3345 got accidentaly merged into it early.

My bad!

@charlielye
Copy link
Contributor Author

Started getting weird conflicts so I just merged downstream #3345 into this as well. So this is become a mega PR :/
You can view the two merged PR's to see those changes in isolation. To see this specific PR's changes in isolation ignoring the downstream merges will probably require a bit of manual git diff.

@charlielye charlielye merged commit 6307e12 into master Nov 18, 2023
76 of 78 checks passed
@charlielye charlielye deleted the cl/all_hashes_in_ts branch November 18, 2023 11:30
rahul-kothari pushed a commit that referenced this pull request Nov 21, 2023
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-packages: 0.15.1</summary>

##
[0.15.1](aztec-packages-v0.15.0...aztec-packages-v0.15.1)
(2023-11-21)


### Features

* **bb:** Add ability to write pk to file or stdout
([#3335](#3335))
([c99862c](c99862c))
* DataBus PoC (UltraHonk as extension of Ultra)
([#3181](#3181))
([dd9dd84](dd9dd84))
* Deploy docs from CCI w/ netlify-cli
([#3348](#3348))
([624d733](624d733))
* Fold batching challenge (alpha)
([#3291](#3291))
([bc99a4f](bc99a4f))
* Open transcript polys as univariates in ECCVM
([#3331](#3331))
([436b22e](436b22e))
* Sandbox packages
([#3360](#3360))
([0dc2d58](0dc2d58))
* Slow updates experimentation
([#2732](#2732))
([193e6c8](193e6c8))
* ZM updates for Translator concatenated polys
([#3343](#3343))
([0e425db](0e425db))


### Bug Fixes

* Bootstrap bbjs.
([#3337](#3337))
([06aedcb](06aedcb))
* Noir-compiler breadth-first resolver
([#3307](#3307))
([02348cf](02348cf))
* Update command looks at devDeps
([#3276](#3276))
([54ee38d](54ee38d)),
closes
[#3275](#3275)
* Updating pedersen benchmarks
([#3211](#3211))
([7e89ff3](7e89ff3))
* Warn on circular imports.
([#3350](#3350))
([5bfbddb](5bfbddb))


### Miscellaneous

* All hashes in ts
([#3333](#3333))
([6307e12](6307e12))
* Compute function tree root in ts.
([#3326](#3326))
([48d8c7f](48d8c7f))
* **docs:** Suggest CLI install per project
([#3267](#3267))
([b4c967b](b4c967b))
* Enforce bracing around blocks. Generally considered easier to read and
less error prone.
([#3349](#3349))
([ee11dec](ee11dec))
* Fix circulars in foundation. Also cleanup fields and optimise to be
buffer underlying.
([#3351](#3351))
([c4bf8d3](c4bf8d3))
* Public kernel tests
([#3325](#3325))
([bace972](bace972))


### Documentation

* Fixed errors in Gas and Fees yellow paper
([#3363](#3363))
([d818206](d818206))
* Initial network section of yellow paper
([#3341](#3341))
([5a18615](5a18615))
* Yellow paper section on Gas and Fees
([#3327](#3327))
([caa7e10](caa7e10))
</details>

<details><summary>barretenberg.js: 0.15.1</summary>

##
[0.15.1](barretenberg.js-v0.15.0...barretenberg.js-v0.15.1)
(2023-11-21)


### Features

* **bb:** Add ability to write pk to file or stdout
([#3335](#3335))
([c99862c](c99862c))


### Miscellaneous

* All hashes in ts
([#3333](#3333))
([6307e12](6307e12))
</details>

<details><summary>barretenberg: 0.15.1</summary>

##
[0.15.1](barretenberg-v0.15.0...barretenberg-v0.15.1)
(2023-11-21)


### Features

* **bb:** Add ability to write pk to file or stdout
([#3335](#3335))
([c99862c](c99862c))
* DataBus PoC (UltraHonk as extension of Ultra)
([#3181](#3181))
([dd9dd84](dd9dd84))
* Fold batching challenge (alpha)
([#3291](#3291))
([bc99a4f](bc99a4f))
* Open transcript polys as univariates in ECCVM
([#3331](#3331))
([436b22e](436b22e))
* ZM updates for Translator concatenated polys
([#3343](#3343))
([0e425db](0e425db))


### Bug Fixes

* Bootstrap bbjs.
([#3337](#3337))
([06aedcb](06aedcb))
* Updating pedersen benchmarks
([#3211](#3211))
([7e89ff3](7e89ff3))


### Miscellaneous

* All hashes in ts
([#3333](#3333))
([6307e12](6307e12))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
AztecBot added a commit to AztecProtocol/barretenberg that referenced this pull request Nov 23, 2023
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-packages: 0.15.1</summary>

##
[0.15.1](AztecProtocol/aztec-packages@aztec-packages-v0.15.0...aztec-packages-v0.15.1)
(2023-11-21)


### Features

* **bb:** Add ability to write pk to file or stdout
([#3335](AztecProtocol/aztec-packages#3335))
([c99862c](AztecProtocol/aztec-packages@c99862c))
* DataBus PoC (UltraHonk as extension of Ultra)
([#3181](AztecProtocol/aztec-packages#3181))
([dd9dd84](AztecProtocol/aztec-packages@dd9dd84))
* Deploy docs from CCI w/ netlify-cli
([#3348](AztecProtocol/aztec-packages#3348))
([624d733](AztecProtocol/aztec-packages@624d733))
* Fold batching challenge (alpha)
([#3291](AztecProtocol/aztec-packages#3291))
([bc99a4f](AztecProtocol/aztec-packages@bc99a4f))
* Open transcript polys as univariates in ECCVM
([#3331](AztecProtocol/aztec-packages#3331))
([436b22e](AztecProtocol/aztec-packages@436b22e))
* Sandbox packages
([#3360](AztecProtocol/aztec-packages#3360))
([0dc2d58](AztecProtocol/aztec-packages@0dc2d58))
* Slow updates experimentation
([#2732](AztecProtocol/aztec-packages#2732))
([193e6c8](AztecProtocol/aztec-packages@193e6c8))
* ZM updates for Translator concatenated polys
([#3343](AztecProtocol/aztec-packages#3343))
([0e425db](AztecProtocol/aztec-packages@0e425db))


### Bug Fixes

* Bootstrap bbjs.
([#3337](AztecProtocol/aztec-packages#3337))
([06aedcb](AztecProtocol/aztec-packages@06aedcb))
* Noir-compiler breadth-first resolver
([#3307](AztecProtocol/aztec-packages#3307))
([02348cf](AztecProtocol/aztec-packages@02348cf))
* Update command looks at devDeps
([#3276](AztecProtocol/aztec-packages#3276))
([54ee38d](AztecProtocol/aztec-packages@54ee38d)),
closes
[#3275](AztecProtocol/aztec-packages#3275)
* Updating pedersen benchmarks
([#3211](AztecProtocol/aztec-packages#3211))
([7e89ff3](AztecProtocol/aztec-packages@7e89ff3))
* Warn on circular imports.
([#3350](AztecProtocol/aztec-packages#3350))
([5bfbddb](AztecProtocol/aztec-packages@5bfbddb))


### Miscellaneous

* All hashes in ts
([#3333](AztecProtocol/aztec-packages#3333))
([6307e12](AztecProtocol/aztec-packages@6307e12))
* Compute function tree root in ts.
([#3326](AztecProtocol/aztec-packages#3326))
([48d8c7f](AztecProtocol/aztec-packages@48d8c7f))
* **docs:** Suggest CLI install per project
([#3267](AztecProtocol/aztec-packages#3267))
([b4c967b](AztecProtocol/aztec-packages@b4c967b))
* Enforce bracing around blocks. Generally considered easier to read and
less error prone.
([#3349](AztecProtocol/aztec-packages#3349))
([ee11dec](AztecProtocol/aztec-packages@ee11dec))
* Fix circulars in foundation. Also cleanup fields and optimise to be
buffer underlying.
([#3351](AztecProtocol/aztec-packages#3351))
([c4bf8d3](AztecProtocol/aztec-packages@c4bf8d3))
* Public kernel tests
([#3325](AztecProtocol/aztec-packages#3325))
([bace972](AztecProtocol/aztec-packages@bace972))


### Documentation

* Fixed errors in Gas and Fees yellow paper
([#3363](AztecProtocol/aztec-packages#3363))
([d818206](AztecProtocol/aztec-packages@d818206))
* Initial network section of yellow paper
([#3341](AztecProtocol/aztec-packages#3341))
([5a18615](AztecProtocol/aztec-packages@5a18615))
* Yellow paper section on Gas and Fees
([#3327](AztecProtocol/aztec-packages#3327))
([caa7e10](AztecProtocol/aztec-packages@caa7e10))
</details>

<details><summary>barretenberg.js: 0.15.1</summary>

##
[0.15.1](AztecProtocol/aztec-packages@barretenberg.js-v0.15.0...barretenberg.js-v0.15.1)
(2023-11-21)


### Features

* **bb:** Add ability to write pk to file or stdout
([#3335](AztecProtocol/aztec-packages#3335))
([c99862c](AztecProtocol/aztec-packages@c99862c))


### Miscellaneous

* All hashes in ts
([#3333](AztecProtocol/aztec-packages#3333))
([6307e12](AztecProtocol/aztec-packages@6307e12))
</details>

<details><summary>barretenberg: 0.15.1</summary>

##
[0.15.1](AztecProtocol/aztec-packages@barretenberg-v0.15.0...barretenberg-v0.15.1)
(2023-11-21)


### Features

* **bb:** Add ability to write pk to file or stdout
([#3335](AztecProtocol/aztec-packages#3335))
([c99862c](AztecProtocol/aztec-packages@c99862c))
* DataBus PoC (UltraHonk as extension of Ultra)
([#3181](AztecProtocol/aztec-packages#3181))
([dd9dd84](AztecProtocol/aztec-packages@dd9dd84))
* Fold batching challenge (alpha)
([#3291](AztecProtocol/aztec-packages#3291))
([bc99a4f](AztecProtocol/aztec-packages@bc99a4f))
* Open transcript polys as univariates in ECCVM
([#3331](AztecProtocol/aztec-packages#3331))
([436b22e](AztecProtocol/aztec-packages@436b22e))
* ZM updates for Translator concatenated polys
([#3343](AztecProtocol/aztec-packages#3343))
([0e425db](AztecProtocol/aztec-packages@0e425db))


### Bug Fixes

* Bootstrap bbjs.
([#3337](AztecProtocol/aztec-packages#3337))
([06aedcb](AztecProtocol/aztec-packages@06aedcb))
* Updating pedersen benchmarks
([#3211](AztecProtocol/aztec-packages#3211))
([7e89ff3](AztecProtocol/aztec-packages@7e89ff3))


### Miscellaneous

* All hashes in ts
([#3333](AztecProtocol/aztec-packages#3333))
([6307e12](AztecProtocol/aztec-packages@6307e12))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants