refactor(avm): replace range and cmp with gadgets #8164

IlyasRidhuan · 2024-08-23T15:48:35Z

Please read contributing guidelines and remove this line.

IlyasRidhuan · 2024-08-23T15:48:49Z

feat(avm): integrate new range and cmp gadgets #8165
refactor(avm): replace range and cmp with gadgets #8164 👈
feat(avm): range check gadget #7967
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @IlyasRidhuan and the rest of your teammates on Graphite

AztecBot · 2024-08-25T10:37:09Z

Benchmark results

Metrics with a significant change:

proof_construction_time_sha256_100_ms (32): 6,519 (+16%)
protocol_circuit_simulation_time_in_ms (private-kernel-tail-to-public): 1,400 (+109%)
avm_simulation_time_ms (Token:transfer_public): 25.6 (-38%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric	1 threads	4 threads	16 threads	32 threads	64 threads
proof_construction_time_sha256_ms	5,856 (+2%)	1,614 (+3%)	752 (+5%)	800 (+3%)	817 (+5%)
proof_construction_time_sha256_30_ms	11,955 (+4%)	3,189 (+3%)	1,459 (+5%)	1,532 (+7%)	1,572 (+7%)
proof_construction_time_sha256_100_ms	45,189 (+2%)	12,378 (+4%)	5,909 (+7%)	⚠️ 6,519 (+16%)	5,878
proof_construction_time_poseidon_hash_ms	79.0 (+1%)	35.0 (+3%)	36.0 (+6%)	60.0 (+3%)	89.0 (+2%)
proof_construction_time_poseidon_hash_30_ms	1,561 (+2%)	430 (+2%)	212 (+5%)	241 (+5%)	279 (+4%)
proof_construction_time_poseidon_hash_100_ms	5,742 (+2%)	1,538 (+1%)	698 (+3%)	750 (+5%)	798 (+7%)

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	4 txs	8 txs	16 txs
l1_rollup_calldata_size_in_bytes	4,356	7,876	14,884
l1_rollup_calldata_gas	50,220	92,984	178,144
l1_rollup_execution_gas	842,880	1,576,540	3,361,756
l2_block_processing_time_in_ms	257 (-2%)	475 (+4%)	805 (-4%)
l2_block_building_time_in_ms	11,436	23,082 (+4%)	44,584
l2_block_rollup_simulation_time_in_ms	11,436	23,082 (+4%)	44,583
l2_block_public_tx_process_time_in_ms	9,764	21,274 (+4%)	42,850

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric	3 blocks	5 blocks
node_history_sync_time_in_ms	3,222 (+6%)	4,342 (+11%)
node_database_size_in_bytes	12,587,088 (-1%)	16,633,936 (-1%)
pxe_database_size_in_bytes	16,254	26,813

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit	simulation_time_in_ms	witness_generation_time_in_ms	input_size_in_bytes	output_size_in_bytes	proving_time_in_ms
private-kernel-init	91.4	395 (-1%)	21,735	44,860	N/A
private-kernel-inner	175 (-1%)	701 (-1%)	72,544	45,007	N/A
private-kernel-reset-tiny	311	715	65,593	44,846	N/A
private-kernel-tail	168 (+1%)	147 (+9%)	50,644	52,257	N/A
base-parity	5.68 (+1%)	N/A	160	96.0	N/A
root-parity	35.6	N/A	73,948	96.0	N/A
base-rollup	3,200 (+4%)	N/A	189,136	664	N/A
block-root-rollup	41.5	N/A	58,205	2,448	N/A
public-kernel-setup	85.5 (+1%)	N/A	105,085	71,222	N/A
public-kernel-app-logic	99.9 (+2%)	N/A	104,911	71,222	N/A
public-kernel-tail	874 (+2%)	N/A	390,582	16,414	N/A
private-kernel-reset-small	312 (+1%)	N/A	66,341	45,629	N/A
private-kernel-tail-to-public	⚠️ 1,400 (+109%)	614	455,400	1,825	N/A
public-kernel-teardown	84.8 (+1%)	N/A	105,349	71,222	N/A
merge-rollup	20.3 (+1%)	N/A	38,174	664	N/A
undefined	N/A	N/A	N/A	N/A	78,550 (+1%)

Stats on running time collected for app circuits

Function	input_size_in_bytes	output_size_in_bytes	witness_generation_time_in_ms
ContractClassRegisterer:register	1,344	11,731	346 (+1%)
ContractInstanceDeployer:deploy	1,408	11,731	18.4
MultiCallEntrypoint:entrypoint	1,920	11,731	408 (+1%)
FeeJuice:deploy	1,376	11,731	395 (+2%)
SchnorrAccount:constructor	1,312	11,731	74.4
SchnorrAccount:entrypoint	2,304	11,731	398 (+1%)
Token:privately_mint_private_note	1,280	11,731	109 (+7%)
FPC:fee_entrypoint_public	1,344	11,731	25.4 (-7%)
Token:transfer	1,312	11,731	226 (+1%)
Benchmarking:create_note	1,344	11,731	86.1
SchnorrAccount:verify_private_authwit	1,280	11,731	27.6
Token:unshield	1,376	11,731	524 (+2%)
FPC:fee_entrypoint_private	1,376	11,731	730 (-3%)

AVM Simulation

Time to simulate various public functions in the AVM.

Function	time_ms	bytecode_size_in_bytes
FeeJuice:_increase_public_balance	59.1 (+1%)	8,174
FeeJuice:set_portal	11.0 (+3%)	4,055
Token:constructor	91.9 (+10%)	29,082
FPC:constructor	59.5 (+8%)	18,940
FeeJuice:mint_public	41.0 (+2%)	6,522
Token:mint_public	46.1 (-8%)	12,704
Token:assert_minter_and_mint	316	8,467
AuthRegistry:set_authorized	38.2 (-18%)	4,194
FPC:prepare_fee	236 (-3%)	6,747
Token:transfer_public	⚠️ 25.6 (-38%)	39,863
FPC:pay_refund	53.8 (-9%)	9,398
Benchmarking:increment_balance	1,241 (+2%)	7,263
Token:_increase_public_balance	43.0 (+4%)	8,686
FPC:pay_refund_with_shielded_rebate	65.4 (+6%)	9,881

Public DB Access

Time to access various public DBs.

Function	time_ms
get-nullifier-index	0.160

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	256 leaves	512 leaves	1024 leaves
batch_insert_into_append_only_tree_16_depth_ms	2.17 (-2%)	3.87 (-1%)	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.8	31.7	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.113 (-3%)	0.109 (-1%)	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	11.4 (+2%)	17.7 (-1%)	30.8 (-3%)	60.5 (+2%)	112 (-1%)
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	95.9	159	287	543	1,055
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.109 (+1%)	0.103 (-1%)	0.100 (-2%)	0.104 (+2%)	0.100 (-1%)
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	14.6	26.1 (+1%)	43.8 (-2%)	86.7 (+5%)	160 (-4%)
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	109	207	355	691	1,363
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.111	0.105	0.106 (+2%)	0.106 (+3%)	0.102 (-2%)
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	16.9 (+1%)	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	132	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	0.107 (+1%)	N/A	N/A	N/A	N/A

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes	1 registered classes
tx_size_in_bytes	64,779	668,997

Transaction size based on fee payment method

| Metric | |
| - | |

jeanmon

LGTM but look at my feedback before merging.

barretenberg/cpp/pil/avm/mem.pil

barretenberg/cpp/src/barretenberg/vm/avm/tests/arithmetic.test.cpp

barretenberg/cpp/src/barretenberg/vm/avm/trace/trace.cpp

barretenberg/cpp/pil/avm/gadgets/range_check.pil

IlyasRidhuan · 2024-08-29T17:34:00Z

Merge activity

Aug 29, 1:34 PM EDT: @IlyasRidhuan started a stack merge that includes this pull request via Graphite.
Aug 29, 1:36 PM EDT: Graphite rebased this pull request as part of a merge.
Aug 29, 2:50 PM EDT: @IlyasRidhuan merged this pull request with Graphite.

🤖 I have created a release *beep* *boop* --- <details><summary>aztec-package: 0.51.1</summary> ## [0.51.1](aztec-package-v0.51.0...aztec-package-v0.51.1) (2024-08-29) ### Features * Add status check to prover agent ([#8248](#8248)) ([7b3006a](7b3006a)) * Faster L1 deployment ([#8234](#8234)) ([51d6699](51d6699)) * Spartan token transfer ([#8163](#8163)) ([38f0157](38f0157)) </details> <details><summary>barretenberg.js: 0.51.1</summary> ## [0.51.1](barretenberg.js-v0.51.0...barretenberg.js-v0.51.1) (2024-08-29) ### Miscellaneous * **barretenberg.js:** Synchronize aztec-packages versions </details> <details><summary>aztec-packages: 0.51.1</summary> ## [0.51.1](aztec-packages-v0.51.0...aztec-packages-v0.51.1) (2024-08-29) ### Features * Add CLI command for gathering proving metrics ([#8221](#8221)) ([5929a42](5929a42)) * Add status check to prover agent ([#8248](#8248)) ([7b3006a](7b3006a)) * **avm:** 1-slot sload/sstore (nr, ts) ([#8264](#8264)) ([bdd9b06](bdd9b06)) * **avm:** Range check gadget ([#7967](#7967)) ([0dd954e](0dd954e)) * **docs:** Add partial notes doc ([#8192](#8192)) ([4299bbd](4299bbd)) * Faster L1 deployment ([#8234](#8234)) ([51d6699](51d6699)) * Initial validator set ([#8133](#8133)) ([6d31ad2](6d31ad2)) * L1-publisher cleanup ([#8148](#8148)) ([6ae2535](6ae2535)) * Proof surgery class ([#8236](#8236)) ([10d7edd](10d7edd)) * Request specific transactions through the p2p layer ([#8185](#8185)) ([54e1cc7](54e1cc7)) * Slot duration flexibility ([#8122](#8122)) ([708e4e5](708e4e5)) * Spartan token transfer ([#8163](#8163)) ([38f0157](38f0157)) ### Bug Fixes * Attempt to fix nightly test ([#8222](#8222)) ([477eec5](477eec5)) * **avm-simulator:** Await avm bytecode check ([#8268](#8268)) ([4410eb3](4410eb3)) * **bb-prover:** Create structure for AVM vk ([#8233](#8233)) ([55b6ba2](55b6ba2)) * **bb:** Mac build ([#8255](#8255)) ([ac54f5c](ac54f5c)) * **ci:** Spot-runner-action was not built ([#8274](#8274)) ([c1509c1](c1509c1)) * **ci:** Try fix brotli edge-case ([#8256](#8256)) ([e03ea0b](e03ea0b)) * Docker containers healthchecks ([#8228](#8228)) ([19edbbb](19edbbb)) * **docs:** Update entrypoint details on accounts page ([#8184](#8184)) ([8453ec7](8453ec7)) * Export brillig names in contract functions ([#8212](#8212)) ([4745741](4745741)) * Fixes for the nightly test run against Sepolia ([#8229](#8229)) ([cfc65c6](cfc65c6)) * Handle constant output for sha256 ([#8251](#8251)) ([0653ba5](0653ba5)) * Log public vm errors as warn in prover-agent ([#8247](#8247)) ([9f4ea9f](9f4ea9f)) * Remove devnet ARM builds for now ([#8202](#8202)) ([81ef715](81ef715)) * Remove fundFpc step from bootstrap ([#8245](#8245)) ([a742531](a742531)) * Ts codegen ([#8267](#8267)) ([cb58800](cb58800)) ### Miscellaneous * Add check to just release images to devnet-deploys ([#8242](#8242)) ([aa6791d](aa6791d)) * Add partial note support for value note ([#8141](#8141)) ([daa57cc](daa57cc)) * Always run `build-check` step in `publish-bb.yml` ([#8240](#8240)) ([5e9749f](5e9749f)) * **avm:** Replace range and cmp with gadgets ([#8164](#8164)) ([cc12558](cc12558)) * Basic network matrix ([#8257](#8257)) ([2a76b1a](2a76b1a)), closes [#8001](#8001) * **bb:** Use std::span in pippenger for scalars ([#8269](#8269)) ([2323cd5](2323cd5)) * Configure interval mining for anvil ([#8211](#8211)) ([eba57b4](eba57b4)) * Create external-ci-approved.yml ([#8235](#8235)) ([24b059b](24b059b)) * Disallow prune in devnet + add onlyOwners ([#8134](#8134)) ([c736f96](c736f96)) * Fix various warnings in noir code ([#8258](#8258)) ([1c6b478](1c6b478)) * Less noisy AVM failures in proving ([#8227](#8227)) ([03bcd62](03bcd62)) * Open an issue if publishing bb fails ([#8223](#8223)) ([2d7a775](2d7a775)) * Reinstate l1-contracts package ([#8250](#8250)) ([263a912](263a912)) * Remove unused generic parameters ([#8249](#8249)) ([00ed045](00ed045)) * Replace relative paths to noir-protocol-circuits ([1783c80](1783c80)) * Replace relative paths to noir-protocol-circuits ([ffe1f35](ffe1f35)) * Report prover metrics ([#8155](#8155)) ([dc7bcdf](dc7bcdf)), closes [#7675](#7675) * Rework balances map ([#8127](#8127)) ([1cac3dd](1cac3dd)), closes [#8104](#8104) * Run CI after merges to provernet ([#8244](#8244)) ([97e5e25](97e5e25)) ### Documentation * Minor fixes ([#8273](#8273)) ([2b8af9e](2b8af9e)) </details> <details><summary>barretenberg: 0.51.1</summary> ## [0.51.1](barretenberg-v0.51.0...barretenberg-v0.51.1) (2024-08-29) ### Features * **avm:** 1-slot sload/sstore (nr, ts) ([#8264](#8264)) ([bdd9b06](bdd9b06)) * **avm:** Range check gadget ([#7967](#7967)) ([0dd954e](0dd954e)) * Proof surgery class ([#8236](#8236)) ([10d7edd](10d7edd)) ### Bug Fixes * **bb-prover:** Create structure for AVM vk ([#8233](#8233)) ([55b6ba2](55b6ba2)) * **bb:** Mac build ([#8255](#8255)) ([ac54f5c](ac54f5c)) * Handle constant output for sha256 ([#8251](#8251)) ([0653ba5](0653ba5)) ### Miscellaneous * **avm:** Replace range and cmp with gadgets ([#8164](#8164)) ([cc12558](cc12558)) * **bb:** Use std::span in pippenger for scalars ([#8269](#8269)) ([2323cd5](2323cd5)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

* master: (28 commits) chore: bump noir-bignum to 0.3.2 (#8276) feat: Populate epoch 0 from initial validator set (#8286) git subrepo push --branch=master noir-projects/aztec-nr git_subrepo.sh: Fix parent in .gitrepo file. [skip ci] chore: replace relative paths to noir-protocol-circuits git subrepo push --branch=master barretenberg chore(master): Release 0.51.1 (#8218) feat(avm): integrate new range and cmp gadgets (#8165) chore: rename process to propose for clarity (#8265) chore: Bump provernet agents to 8 and speed up bot (#8280) feat(avm): avm recursive verifier cpp (#8162) feat: moving fee payout + make proof submission sequential (#8262) feat: Block cheat codes in anvil (#8277) feat: wallet tx management (#8246) refactor(avm): replace range and cmp with gadgets (#8164) fix(docs): Update entrypoint details on accounts page (#8184) feat: l1-publisher cleanup (#8148) feat(avm): range check gadget (#7967) fix(ci): spot-runner-action was not built (#8274) fix: ts codegen (#8267) ...

Please read [contributing guidelines](CONTRIBUTING.md) and remove this line.

IlyasRidhuan mentioned this pull request Aug 23, 2024

feat(avm): range check gadget #7967

Merged

IlyasRidhuan mentioned this pull request Aug 23, 2024

feat(avm): integrate new range and cmp gadgets #8165

Merged

IlyasRidhuan changed the title ~~refactor(avm): extract rng chk from gas and mem~~ refactor(avm): replace range and cmp with gadgets Aug 23, 2024

IlyasRidhuan force-pushed the ir/08-06-feat_avm_range_check_gadget branch from 7322530 to 32fef12 Compare August 23, 2024 16:37

IlyasRidhuan force-pushed the ir/08-14-feat_avm_extract_rng_to_gadget branch from 93b1914 to 8ada417 Compare August 23, 2024 16:37

IlyasRidhuan marked this pull request as ready for review August 25, 2024 09:58

IlyasRidhuan requested review from jeanmon, fcarreiro and Maddiaa0 as code owners August 25, 2024 09:58

IlyasRidhuan force-pushed the ir/08-14-feat_avm_extract_rng_to_gadget branch from 8ada417 to 543aa2d Compare August 25, 2024 19:01

jeanmon approved these changes Aug 26, 2024

View reviewed changes

IlyasRidhuan force-pushed the ir/08-06-feat_avm_range_check_gadget branch from 32fef12 to ee25e4f Compare August 28, 2024 12:14

IlyasRidhuan force-pushed the ir/08-14-feat_avm_extract_rng_to_gadget branch from 543aa2d to a5d24a0 Compare August 28, 2024 12:14

IlyasRidhuan force-pushed the ir/08-06-feat_avm_range_check_gadget branch from ee25e4f to 332f078 Compare August 28, 2024 13:10

IlyasRidhuan force-pushed the ir/08-14-feat_avm_extract_rng_to_gadget branch from a5d24a0 to b22d58e Compare August 28, 2024 13:11

IlyasRidhuan force-pushed the ir/08-06-feat_avm_range_check_gadget branch from 332f078 to 534ec0a Compare August 28, 2024 16:01

IlyasRidhuan force-pushed the ir/08-14-feat_avm_extract_rng_to_gadget branch from b22d58e to 9915be7 Compare August 28, 2024 16:01

IlyasRidhuan force-pushed the ir/08-06-feat_avm_range_check_gadget branch from 534ec0a to 0ea2110 Compare August 29, 2024 09:40

IlyasRidhuan force-pushed the ir/08-14-feat_avm_extract_rng_to_gadget branch from 9915be7 to cf5d27f Compare August 29, 2024 10:57

IlyasRidhuan changed the base branch from ir/08-06-feat_avm_range_check_gadget to graphite-base/8164 August 29, 2024 17:34

IlyasRidhuan changed the base branch from graphite-base/8164 to master August 29, 2024 17:34

refactor(avm): extract rng chk from gas and mem

d4f0fcc

IlyasRidhuan force-pushed the ir/08-14-feat_avm_extract_rng_to_gadget branch from cf5d27f to d4f0fcc Compare August 29, 2024 17:35

IlyasRidhuan merged commit cc12558 into master Aug 29, 2024
101 checks passed

IlyasRidhuan deleted the ir/08-14-feat_avm_extract_rng_to_gadget branch August 29, 2024 18:50

AztecBot mentioned this pull request Aug 29, 2024

chore(master): Release 0.51.1 #8218

Merged

codygunton pushed a commit that referenced this pull request Aug 30, 2024

refactor(avm): replace range and cmp with gadgets (#8164)

855bc2b

Please read [contributing guidelines](CONTRIBUTING.md) and remove this line.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(avm): replace range and cmp with gadgets #8164

refactor(avm): replace range and cmp with gadgets #8164

IlyasRidhuan commented Aug 23, 2024

IlyasRidhuan commented Aug 23, 2024 •

edited

Loading

AztecBot commented Aug 25, 2024 •

edited

Loading

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

AVM Simulation

Public DB Access

Tree insertion stats

Miscellaneous

jeanmon left a comment

IlyasRidhuan commented Aug 29, 2024 •

edited

Loading

refactor(avm): replace range and cmp with gadgets #8164

refactor(avm): replace range and cmp with gadgets #8164

Conversation

IlyasRidhuan commented Aug 23, 2024

IlyasRidhuan commented Aug 23, 2024 • edited Loading

AztecBot commented Aug 25, 2024 • edited Loading

Benchmark results

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

AVM Simulation

Public DB Access

Tree insertion stats

Miscellaneous

jeanmon left a comment

Choose a reason for hiding this comment

IlyasRidhuan commented Aug 29, 2024 • edited Loading

Merge activity

IlyasRidhuan commented Aug 23, 2024 •

edited

Loading

AztecBot commented Aug 25, 2024 •

edited

Loading

IlyasRidhuan commented Aug 29, 2024 •

edited

Loading