Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: serialize AvmVerificationKeyData #8529

Merged
merged 3 commits into from
Sep 13, 2024
Merged

Conversation

alexghr
Copy link
Contributor

@alexghr alexghr commented Sep 13, 2024

This PR adds a integration test for the RPC layer between the queue and agent to validate that all the function calls correctly encode/decode their inputs and outputs.

@alexghr alexghr force-pushed the ag/fix-serialisation-error branch 2 times, most recently from c567242 to be26d4d Compare September 13, 2024 10:54
@alexghr alexghr enabled auto-merge (squash) September 13, 2024 10:54
@AztecBot
Copy link
Collaborator

AztecBot commented Sep 13, 2024

Benchmark results

Metrics with a significant change:

  • l2_block_processing_time_in_ms (4): 257 (-27%)
  • l2_block_processing_time_in_ms (16): 821 (-29%)
Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric 1 threads 4 threads 16 threads 32 threads 64 threads
proof_construction_time_sha256_ms 5,762 1,546 700 747 (+1%) 770 (-1%)
proof_construction_time_sha256_30_ms 11,527 3,066 1,369 1,413 (-1%) 1,458
proof_construction_time_sha256_100_ms 45,379 11,896 5,397 5,381 (-1%) 5,340
proof_construction_time_poseidon_hash_ms 79.0 34.0 34.0 58.0 (-2%) 88.0 (+1%)
proof_construction_time_poseidon_hash_30_ms 1,535 417 199 (-1%) 228 (+3%) 274 (+3%)
proof_construction_time_poseidon_hash_100_ms 5,672 1,506 671 735 (-2%) 750 (+1%)

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 4 txs 8 txs 16 txs
l1_rollup_calldata_size_in_bytes 4,612 8,260 15,524
l1_rollup_calldata_gas 52,732 97,550 186,836
l1_rollup_execution_gas 819,418 1,555,644 3,346,365
l2_block_processing_time_in_ms ⚠️ 257 (-27%) 454 (-2%) ⚠️ 821 (-29%)
l2_block_building_time_in_ms 9,217 18,103 36,271
l2_block_rollup_simulation_time_in_ms 9,217 18,103 36,271
l2_block_public_tx_process_time_in_ms 7,842 16,711 34,829

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric 3 blocks 5 blocks
node_history_sync_time_in_ms 3,145 (+2%) 4,036 (+2%)
node_database_size_in_bytes 14,225,648 18,182,384
pxe_database_size_in_bytes 16,258 26,818

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit simulation_time_in_ms witness_generation_time_in_ms input_size_in_bytes output_size_in_bytes proving_time_in_ms
private-kernel-init 70.4 (+1%) 372 (-2%) 20,971 44,933 N/A
private-kernel-inner 138 688 71,782 45,067 N/A
private-kernel-reset-tiny 322 (+2%) 683 64,268 44,922 N/A
private-kernel-tail 150 128 (-1%) 50,548 57,178 N/A
base-parity 6.02 N/A 160 96.0 N/A
root-parity 35.5 (+1%) N/A 73,964 96.0 N/A
base-rollup 2,092 (+1%) N/A 189,128 664 N/A
block-root-rollup 42.0 (-1%) N/A 58,217 2,448 N/A
public-kernel-setup 85.9 (+1%) N/A 114,433 79,670 N/A
public-kernel-app-logic 98.1 N/A 114,251 79,670 N/A
public-kernel-tail 564 (-1%) N/A 487,098 16,414 N/A
private-kernel-reset-small 288 N/A 66,345 45,629 N/A
private-kernel-tail-to-public 793 (+1%) 591 447,469 1,825 N/A
public-kernel-teardown 86.0 (+1%) N/A 114,697 79,670 N/A
merge-rollup 20.4 (+1%) N/A 38,182 664 N/A
undefined N/A N/A N/A N/A 105,016 (-5%)

Stats on running time collected for app circuits

Function input_size_in_bytes output_size_in_bytes witness_generation_time_in_ms
ContractClassRegisterer:register 1,344 11,731 345
ContractInstanceDeployer:deploy 1,408 11,731 19.1 (+1%)
MultiCallEntrypoint:entrypoint 1,920 11,731 425 (+1%)
FeeJuice:deploy 1,376 11,731 395 (-1%)
SchnorrAccount:constructor 1,312 11,731 64.0 (+1%)
SchnorrAccount:entrypoint 2,336 11,731 372
FeeJuice:claim 1,344 11,731 35.5 (-6%)
Token:privately_mint_private_note 1,280 11,731 73.2
FPC:fee_entrypoint_public 1,344 11,731 24.2 (-2%)
Token:transfer 1,312 11,731 210 (+1%)
Benchmarking:create_note 1,344 11,731 77.6
SchnorrAccount:verify_private_authwit 1,280 11,731 27.3 (+2%)
Token:unshield 1,376 11,731 519 (+5%)
FPC:fee_entrypoint_private 1,376 11,731 687 (+6%)

AVM Simulation

Time to simulate various public functions in the AVM.

Function time_ms bytecode_size_in_bytes
FeeJuice:_increase_public_balance 42.7 (-2%) 1,197
FeeJuice:set_portal 14.1 (+16%) 1,179
Token:constructor 280 (-1%) 10,585
FPC:constructor 71.4 (-2%) 7,987
FeeJuice:check_balance 37.2 (+1%) 1,004
Token:mint_public 46.0 (+2%) 2,122
Token:assert_minter_and_mint 242 1,434
AuthRegistry:set_authorized 45.6 (+1%) 501
FPC:prepare_fee 105 (+20%) 1,379
Token:transfer_public 24.5 (+20%) 8,730
FPC:pay_refund 45.3 (+7%) 1,915
Benchmarking:increment_balance 985 1,575
Token:_increase_public_balance 40.1 (+6%) 1,168
FPC:pay_refund_with_shielded_rebate 51.3 1,915

Public DB Access

Time to access various public DBs.

Function time_ms
get-nullifier-index 0.163

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric 1 leaves 16 leaves 64 leaves 128 leaves 256 leaves 512 leaves 1024 leaves
batch_insert_into_append_only_tree_16_depth_ms 2.21 (+2%) 3.91 (+2%) N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_count 16.7 31.8 N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_ms 0.116 (+2%) 0.110 (+2%) N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_32_depth_ms N/A N/A 11.3 (+1%) 18.0 (+1%) 32.1 (+2%) 61.2 (+4%) 116
batch_insert_into_append_only_tree_32_depth_hash_count N/A N/A 95.9 159 287 543 1,055
batch_insert_into_append_only_tree_32_depth_hash_ms N/A N/A 0.108 (+1%) 0.105 (+1%) 0.104 (+2%) 0.105 (+3%) 0.104
batch_insert_into_indexed_tree_20_depth_ms N/A N/A 14.4 (+1%) 26.7 (+1%) 44.7 84.8 (+4%) 169 (+3%)
batch_insert_into_indexed_tree_20_depth_hash_count N/A N/A 109 207 357 691 1,363
batch_insert_into_indexed_tree_20_depth_hash_ms N/A N/A 0.110 (+1%) 0.107 (+1%) 0.107 (-1%) 0.105 (+4%) 0.108 (+3%)
batch_insert_into_indexed_tree_40_depth_ms N/A N/A 16.5 (+3%) N/A N/A N/A N/A
batch_insert_into_indexed_tree_40_depth_hash_count N/A N/A 129 N/A N/A N/A N/A
batch_insert_into_indexed_tree_40_depth_hash_ms N/A N/A 0.108 (+2%) N/A N/A N/A N/A

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric 0 registered classes 1 registered classes
tx_size_in_bytes 72,419 668,302

Transaction size based on fee payment method

| Metric | |
| - | |

@alexghr alexghr force-pushed the ag/fix-serialisation-error branch from be26d4d to fca596b Compare September 13, 2024 12:33
@alexghr alexghr force-pushed the ag/fix-serialisation-error branch from fca596b to 497d37f Compare September 13, 2024 14:00
@alexghr alexghr merged commit 78c94a4 into master Sep 13, 2024
48 checks passed
@alexghr alexghr deleted the ag/fix-serialisation-error branch September 13, 2024 16:01
spypsy pushed a commit that referenced this pull request Sep 17, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.55.1</summary>

##
[0.55.1](aztec-package-v0.55.0...aztec-package-v0.55.1)
(2024-09-17)


### Features

* CI deploy on sepolia
([#8514](#8514))
([54f0344](54f0344))
* Tx verification & peer scoring on p2p layer. bot support for
EasyPrivateToken
([#8298](#8298))
([beb651f](beb651f))


### Miscellaneous

* Remove ARCHIVER_L1_START_BLOCK
([#8554](#8554))
([bc8d461](bc8d461))
</details>

<details><summary>barretenberg.js: 0.55.1</summary>

##
[0.55.1](barretenberg.js-v0.55.0...barretenberg.js-v0.55.1)
(2024-09-17)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.55.1</summary>

##
[0.55.1](aztec-packages-v0.55.0...aztec-packages-v0.55.1)
(2024-09-17)


### Features

* `TXE::store_note_in_cache` --&gt; `TXE::add_note`
([#8547](#8547))
([5a6aaeb](5a6aaeb))
* Add a `comptime` string type for string handling at compile-time
(noir-lang/noir#6026)
([cd7983a](cd7983a))
* CI deploy on sepolia
([#8514](#8514))
([54f0344](54f0344))
* Default to outputting witness with file named after package
(noir-lang/noir#6031)
([cd7983a](cd7983a))
* Let LSP suggest trait impl methods as you are typing them
(noir-lang/noir#6029)
([cd7983a](cd7983a))
* NFT with "transient" storage shield flow
([#8129](#8129))
([578f67c](578f67c))
* Optimize allocating immediate amounts of memory
([#8579](#8579))
([e0185e7](e0185e7))
* Spartan iac
([#8455](#8455))
([16fba46](16fba46))
* Sync from aztec-packages (noir-lang/noir#6028)
([cd7983a](cd7983a))
* Tx verification & peer scoring on p2p layer. bot support for
EasyPrivateToken
([#8298](#8298))
([beb651f](beb651f))
* Unconstraining keys in unconstrained encryption
([#7912](#7912))
([eb9275a](eb9275a))
* Update args hash to be a flat poseidon
([#8571](#8571))
([0c54224](0c54224))
* Use poseidon for fn selectors
([#8239](#8239))
([41891db](41891db))


### Bug Fixes

* Disable side-effects for no_predicates functions
(noir-lang/noir#6027)
([cd7983a](cd7983a))
* Native world state test issues
([#8546](#8546))
([aab8773](aab8773))
* Remove special case for epoch 0
([#8549](#8549))
([b035d01](b035d01))
* Serialize AvmVerificationKeyData
([#8529](#8529))
([78c94a4](78c94a4))


### Miscellaneous

* 7791: Disable world_state test suite
([#8594](#8594))
([ee21583](ee21583))
* Add jq to aztec image
([#8542](#8542))
([a7fb791](a7fb791))
* Add sync suite
([#8550](#8550))
([ce0a9db](ce0a9db))
* **ci:** Action to redo typo PRs
([#8553](#8553))
([3ed5879](3ed5879))
* **ci:** Fix master
([#8534](#8534))
([47c368f](47c368f))
* **ci:** Fix redo-typo-pr.yml
([abf9802](abf9802))
* **ci:** Fix redo-typo-pr.yml
([#8555](#8555))
([7f1673c](7f1673c))
* **ci:** Hotfix
([ffd31aa](ffd31aa))
* **ci:** Hotfix arm ci
([979f267](979f267))
* **ci:** Optimize disk usage in arm run
([#8564](#8564))
([33e6aa4](33e6aa4))
* **ci:** Use labels and if branch=master to control jobs
([#8508](#8508))
([68a2226](68a2226))
* GitHub Actions Deployments to Amazon EKS
([#8563](#8563))
([6fae8f0](6fae8f0))
* Moves add gate out of aux
([#8541](#8541))
([c3ad163](c3ad163))
* Protogalaxy verifier matches prover 2
([#8477](#8477))
([58882b1](58882b1))
* Redo typo PR by ankushgoel27
([#8595](#8595))
([7ca6d24](7ca6d24))
* Redo typo PR by Ocheretovich
([#8559](#8559))
([c4296ba](c4296ba))
* Redo typo PR by Olexandr88
([#8560](#8560))
([e35d148](e35d148))
* Redo typo PR by skaunov
([#8557](#8557))
([8a1e7c3](8a1e7c3))
* Release Noir(0.34.0) (noir-lang/noir#5692)
([cd7983a](cd7983a))
* Remove ARCHIVER_L1_START_BLOCK
([#8554](#8554))
([bc8d461](bc8d461))
* Remove redundant e2e tests and organize
([#8561](#8561))
([de2b775](de2b775))
* Remove unused imports
([#8556](#8556))
([e11242e](e11242e))
* Replace relative paths to noir-protocol-circuits
([2336986](2336986))
* Replace relative paths to noir-protocol-circuits
([9668ed5](9668ed5))
</details>

<details><summary>barretenberg: 0.55.1</summary>

##
[0.55.1](barretenberg-v0.55.0...barretenberg-v0.55.1)
(2024-09-17)


### Bug Fixes

* Native world state test issues
([#8546](#8546))
([aab8773](aab8773))


### Miscellaneous

* 7791: Disable world_state test suite
([#8594](#8594))
([ee21583](ee21583))
* Moves add gate out of aux
([#8541](#8541))
([c3ad163](c3ad163))
* Protogalaxy verifier matches prover 2
([#8477](#8477))
([58882b1](58882b1))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
AztecBot added a commit to AztecProtocol/barretenberg that referenced this pull request Sep 18, 2024
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.55.1</summary>

##
[0.55.1](AztecProtocol/aztec-packages@aztec-package-v0.55.0...aztec-package-v0.55.1)
(2024-09-17)


### Features

* CI deploy on sepolia
([#8514](AztecProtocol/aztec-packages#8514))
([54f0344](AztecProtocol/aztec-packages@54f0344))
* Tx verification & peer scoring on p2p layer. bot support for
EasyPrivateToken
([#8298](AztecProtocol/aztec-packages#8298))
([beb651f](AztecProtocol/aztec-packages@beb651f))


### Miscellaneous

* Remove ARCHIVER_L1_START_BLOCK
([#8554](AztecProtocol/aztec-packages#8554))
([bc8d461](AztecProtocol/aztec-packages@bc8d461))
</details>

<details><summary>barretenberg.js: 0.55.1</summary>

##
[0.55.1](AztecProtocol/aztec-packages@barretenberg.js-v0.55.0...barretenberg.js-v0.55.1)
(2024-09-17)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.55.1</summary>

##
[0.55.1](AztecProtocol/aztec-packages@aztec-packages-v0.55.0...aztec-packages-v0.55.1)
(2024-09-17)


### Features

* `TXE::store_note_in_cache` --&gt; `TXE::add_note`
([#8547](AztecProtocol/aztec-packages#8547))
([5a6aaeb](AztecProtocol/aztec-packages@5a6aaeb))
* Add a `comptime` string type for string handling at compile-time
(noir-lang/noir#6026)
([cd7983a](AztecProtocol/aztec-packages@cd7983a))
* CI deploy on sepolia
([#8514](AztecProtocol/aztec-packages#8514))
([54f0344](AztecProtocol/aztec-packages@54f0344))
* Default to outputting witness with file named after package
(noir-lang/noir#6031)
([cd7983a](AztecProtocol/aztec-packages@cd7983a))
* Let LSP suggest trait impl methods as you are typing them
(noir-lang/noir#6029)
([cd7983a](AztecProtocol/aztec-packages@cd7983a))
* NFT with "transient" storage shield flow
([#8129](AztecProtocol/aztec-packages#8129))
([578f67c](AztecProtocol/aztec-packages@578f67c))
* Optimize allocating immediate amounts of memory
([#8579](AztecProtocol/aztec-packages#8579))
([e0185e7](AztecProtocol/aztec-packages@e0185e7))
* Spartan iac
([#8455](AztecProtocol/aztec-packages#8455))
([16fba46](AztecProtocol/aztec-packages@16fba46))
* Sync from aztec-packages (noir-lang/noir#6028)
([cd7983a](AztecProtocol/aztec-packages@cd7983a))
* Tx verification & peer scoring on p2p layer. bot support for
EasyPrivateToken
([#8298](AztecProtocol/aztec-packages#8298))
([beb651f](AztecProtocol/aztec-packages@beb651f))
* Unconstraining keys in unconstrained encryption
([#7912](AztecProtocol/aztec-packages#7912))
([eb9275a](AztecProtocol/aztec-packages@eb9275a))
* Update args hash to be a flat poseidon
([#8571](AztecProtocol/aztec-packages#8571))
([0c54224](AztecProtocol/aztec-packages@0c54224))
* Use poseidon for fn selectors
([#8239](AztecProtocol/aztec-packages#8239))
([41891db](AztecProtocol/aztec-packages@41891db))


### Bug Fixes

* Disable side-effects for no_predicates functions
(noir-lang/noir#6027)
([cd7983a](AztecProtocol/aztec-packages@cd7983a))
* Native world state test issues
([#8546](AztecProtocol/aztec-packages#8546))
([aab8773](AztecProtocol/aztec-packages@aab8773))
* Remove special case for epoch 0
([#8549](AztecProtocol/aztec-packages#8549))
([b035d01](AztecProtocol/aztec-packages@b035d01))
* Serialize AvmVerificationKeyData
([#8529](AztecProtocol/aztec-packages#8529))
([78c94a4](AztecProtocol/aztec-packages@78c94a4))


### Miscellaneous

* 7791: Disable world_state test suite
([#8594](AztecProtocol/aztec-packages#8594))
([ee21583](AztecProtocol/aztec-packages@ee21583))
* Add jq to aztec image
([#8542](AztecProtocol/aztec-packages#8542))
([a7fb791](AztecProtocol/aztec-packages@a7fb791))
* Add sync suite
([#8550](AztecProtocol/aztec-packages#8550))
([ce0a9db](AztecProtocol/aztec-packages@ce0a9db))
* **ci:** Action to redo typo PRs
([#8553](AztecProtocol/aztec-packages#8553))
([3ed5879](AztecProtocol/aztec-packages@3ed5879))
* **ci:** Fix master
([#8534](AztecProtocol/aztec-packages#8534))
([47c368f](AztecProtocol/aztec-packages@47c368f))
* **ci:** Fix redo-typo-pr.yml
([abf9802](AztecProtocol/aztec-packages@abf9802))
* **ci:** Fix redo-typo-pr.yml
([#8555](AztecProtocol/aztec-packages#8555))
([7f1673c](AztecProtocol/aztec-packages@7f1673c))
* **ci:** Hotfix
([ffd31aa](AztecProtocol/aztec-packages@ffd31aa))
* **ci:** Hotfix arm ci
([979f267](AztecProtocol/aztec-packages@979f267))
* **ci:** Optimize disk usage in arm run
([#8564](AztecProtocol/aztec-packages#8564))
([33e6aa4](AztecProtocol/aztec-packages@33e6aa4))
* **ci:** Use labels and if branch=master to control jobs
([#8508](AztecProtocol/aztec-packages#8508))
([68a2226](AztecProtocol/aztec-packages@68a2226))
* GitHub Actions Deployments to Amazon EKS
([#8563](AztecProtocol/aztec-packages#8563))
([6fae8f0](AztecProtocol/aztec-packages@6fae8f0))
* Moves add gate out of aux
([#8541](AztecProtocol/aztec-packages#8541))
([c3ad163](AztecProtocol/aztec-packages@c3ad163))
* Protogalaxy verifier matches prover 2
([#8477](AztecProtocol/aztec-packages#8477))
([58882b1](AztecProtocol/aztec-packages@58882b1))
* Redo typo PR by ankushgoel27
([#8595](AztecProtocol/aztec-packages#8595))
([7ca6d24](AztecProtocol/aztec-packages@7ca6d24))
* Redo typo PR by Ocheretovich
([#8559](AztecProtocol/aztec-packages#8559))
([c4296ba](AztecProtocol/aztec-packages@c4296ba))
* Redo typo PR by Olexandr88
([#8560](AztecProtocol/aztec-packages#8560))
([e35d148](AztecProtocol/aztec-packages@e35d148))
* Redo typo PR by skaunov
([#8557](AztecProtocol/aztec-packages#8557))
([8a1e7c3](AztecProtocol/aztec-packages@8a1e7c3))
* Release Noir(0.34.0) (noir-lang/noir#5692)
([cd7983a](AztecProtocol/aztec-packages@cd7983a))
* Remove ARCHIVER_L1_START_BLOCK
([#8554](AztecProtocol/aztec-packages#8554))
([bc8d461](AztecProtocol/aztec-packages@bc8d461))
* Remove redundant e2e tests and organize
([#8561](AztecProtocol/aztec-packages#8561))
([de2b775](AztecProtocol/aztec-packages@de2b775))
* Remove unused imports
([#8556](AztecProtocol/aztec-packages#8556))
([e11242e](AztecProtocol/aztec-packages@e11242e))
* Replace relative paths to noir-protocol-circuits
([2336986](AztecProtocol/aztec-packages@2336986))
* Replace relative paths to noir-protocol-circuits
([9668ed5](AztecProtocol/aztec-packages@9668ed5))
</details>

<details><summary>barretenberg: 0.55.1</summary>

##
[0.55.1](AztecProtocol/aztec-packages@barretenberg-v0.55.0...barretenberg-v0.55.1)
(2024-09-17)


### Bug Fixes

* Native world state test issues
([#8546](AztecProtocol/aztec-packages#8546))
([aab8773](AztecProtocol/aztec-packages@aab8773))


### Miscellaneous

* 7791: Disable world_state test suite
([#8594](AztecProtocol/aztec-packages#8594))
([ee21583](AztecProtocol/aztec-packages@ee21583))
* Moves add gate out of aux
([#8541](AztecProtocol/aztec-packages#8541))
([c3ad163](AztecProtocol/aztec-packages@c3ad163))
* Protogalaxy verifier matches prover 2
([#8477](AztecProtocol/aztec-packages#8477))
([58882b1](AztecProtocol/aztec-packages@58882b1))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants