feat: modular CLI + `aztec test` #7426

Thunkar · 2024-07-10T15:18:13Z

Refactors the CLI package to work as a collection of dynamically importable commands in actual CLI programs, such as aztec.

Right now this doesn't add that much value besides avoiding code repetition between builder and CLI, but tidies up the packages and makes it so much easier to build different utilities by mix and matching different commands. This is going to be very useful when we strip the aztec binary of some internal commands, and even more so when we start building the infamous CLI wallet (which will certainly pull some of these commands in)

Besides that, it cleans up the sandbox so that no redundant aztecprotocol/cli image is pulled, and some of the commands are greatly simplified. The sandbox now does:

aztec: All the previous commands + all the old CLI ones without having to prefix them with cli
aztec-nargo: Same as always

REMOVED:

aztec-sandbox and aztec sandbox: now aztec start --sandbox
aztec-builder: now aztec codegen + aztec update

ADDED:

aztec test: runs aztec start --txe&& aztec-nargo test --use-legacy --oracle-resolver http://aztec:8081 --silence-warnings via docker-compose allowing users to easily run contract tests using TXE

… gj/builder-to-cli

AztecBot · 2024-07-10T15:54:07Z

Benchmark results

Metrics with a significant change:

avm_simulation_time_ms (Token:assert_minter_and_mint): 83.5 (+70%)
avm_simulation_time_ms (Token:transfer_public): 64.3 (+49%)
protocol_circuit_proving_time_in_ms (root-parity): 32,688 (-35%)
protocol_circuit_proving_time_in_ms (public-kernel-setup): 18,650 (-16%)
protocol_circuit_proving_time_in_ms (root-rollup): 30,442 (-25%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric	1 threads	4 threads	16 threads	32 threads	64 threads
proof_construction_time_sha256_ms	5,741 (+7%)	1,545 (+5%)	703 (+6%)	742 (+2%)	770 (+6%)
proof_construction_time_sha256_30_ms	11,759 (+8%)	3,158 (+7%)	1,424 (+9%)	1,435 (+5%)	1,470 (+5%)
proof_construction_time_sha256_100_ms	43,967 (+6%)	11,892 (+5%)	5,572 (+10%)	5,508 (-3%)	5,406 (+6%)
proof_construction_time_poseidon_hash_ms	78.0 (+4%)	34.0	34.0	60.0 (+5%)	88.0 (-1%)
proof_construction_time_poseidon_hash_30_ms	1,517 (+5%)	415 (+3%)	201 (+4%)	228 (+7%)	264 (+1%)
proof_construction_time_poseidon_hash_100_ms	5,625 (+5%)	1,517 (+4%)	679 (+6%)	748 (+9%)	749 (+6%)

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	4 txs	8 txs	16 txs
l1_rollup_calldata_size_in_bytes	708	708	708
l1_rollup_calldata_gas	6,600 (+1%)	6,600	6,588
l1_rollup_execution_gas	583,062	583,213	583,352
l2_block_processing_time_in_ms	776 (-4%)	1,423 (-5%)	2,729 (-6%)
l2_block_building_time_in_ms	13,131 (-5%)	26,015 (-4%)	50,422 (-5%)
l2_block_rollup_simulation_time_in_ms	13,130 (-5%)	26,015 (-4%)	50,421 (-5%)
l2_block_public_tx_process_time_in_ms	11,003 (-5%)	23,651 (-4%)	48,057 (-5%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric	3 blocks	5 blocks
node_history_sync_time_in_ms	7,086 (-5%)	9,889 (-5%)
node_database_size_in_bytes	12,394,576	16,408,656
pxe_database_size_in_bytes	16,254	26,813

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit	simulation_time_in_ms	witness_generation_time_in_ms	input_size_in_bytes	output_size_in_bytes	proving_time_in_ms	proof_size_in_bytes	num_public_inputs	size_in_gates
private-kernel-init	117 (-3%)	400 (+1%)	21,732	54,085	N/A	N/A	N/A	N/A
private-kernel-inner	230 (-3%)	721 (+2%)	81,199	54,264	N/A	N/A	N/A	N/A
private-kernel-tail	1,126 (-1%)	2,555 (+2%)	61,808	61,554	N/A	N/A	N/A	N/A
base-parity	6.47 (-4%)	844 (-2%)	160	96.0	2,153 (+8%)	12,676	3.00	131,072
root-parity	108 (-3%)	125 (+7%)	64,860	96.0	⚠️ 32,688 (-35%)	12,676	3.00	4,194,304
base-rollup	4,211 (-4%)	5,157 (+5%)	180,315	632	46,383 (-4%)	13,476	28.0	4,194,304
root-rollup	153 (-5%)	124 (+2%)	51,261	652	⚠️ 30,442 (-25%)	13,412	26.0	4,194,304
public-kernel-setup	195 (-4%)	2,535 (+4%)	112,536	80,310	⚠️ 18,650 (-16%)	118,916	3,323	2,097,152
public-kernel-app-logic	150 (-3%)	3,506 (+8%)	112,536	80,310	11,026 (+10%)	118,916	3,323	1,048,576
public-kernel-tail	892 (-3%)	25,257 (-7%)	409,413	10,046	73,088	24,868	384	8,388,608
private-kernel-reset-tiny	299 (-4%)	1,009 (+1%)	77,007	53,954	N/A	N/A	N/A	N/A
private-kernel-tail-to-public	6,954 (+2%)	2,862 (+1%)	777,415	1,632	N/A	N/A	N/A	N/A
public-kernel-teardown	139 (-4%)	3,494 (+7%)	112,536	80,310	19,144 (-15%)	118,916	3,323	2,097,152
merge-rollup	57.4 (-6%)	N/A	33,566	632	N/A	N/A	N/A	N/A
undefined	N/A	N/A	N/A	N/A	182,275 (+13%)	N/A	N/A	N/A

Stats on running time collected for app circuits

Function	input_size_in_bytes	output_size_in_bytes	witness_generation_time_in_ms	proof_size_in_bytes	proving_time_in_ms
ContractClassRegisterer:register	1,312	9,344	402 (-2%)	N/A	N/A
ContractInstanceDeployer:deploy	1,376	9,344	25.2 (-4%)	N/A	N/A
MultiCallEntrypoint:entrypoint	1,888	9,344	635 (-3%)	N/A	N/A
GasToken:deploy	1,344	9,344	588	N/A	N/A
SchnorrAccount:constructor	1,280	9,344	431 (-6%)	N/A	N/A
SchnorrAccount:entrypoint	2,272	9,344	769 (-6%)	N/A	N/A
Token:privately_mint_private_note	1,248	9,344	498 (-6%)	N/A	N/A
FPC:fee_entrypoint_public	1,312	9,344	100 (-3%)	N/A	N/A
Token:transfer	1,280	9,344	1,467 (-5%)	N/A	N/A
AuthRegistry:set_authorized (avm)	19,222	N/A	N/A	95,392	2,156 (+2%)
FPC:prepare_fee (avm)	26,664	N/A	N/A	95,456	2,938 (+4%)
Token:transfer_public (avm)	42,914	N/A	N/A	95,456	3,383 (+6%)
AuthRegistry:consume (avm)	33,100	N/A	N/A	95,424	2,852 (+1%)
FPC:pay_refund (avm)	36,829	N/A	N/A	95,424	11,745 (+7%)
Benchmarking:create_note	1,312	9,344	426 (-9%)	N/A	N/A
SchnorrAccount:verify_private_authwit	1,248	9,344	41.6 (-10%)	N/A	N/A
Token:unshield	1,344	9,344	1,189 (-7%)	N/A	N/A
FPC:fee_entrypoint_private	1,344	9,344	1,556 (-7%)	N/A	N/A

AVM Simulation

Time to simulate various public functions in the AVM.

Function	time_ms	bytecode_size_in_bytes
GasToken:_increase_public_balance	100 (+1%)	13,790
GasToken:set_portal	12.2 (-6%)	3,339
Token:constructor	122 (-5%)	23,692
FPC:constructor	93.4 (+2%)	13,592
GasToken:mint_public	81.6 (-1%)	10,158
Token:mint_public	345 (-1%)	19,034
Token:assert_minter_and_mint	⚠️ 83.5 (+70%)	12,925
AuthRegistry:set_authorized	48.6 (+1%)	7,812
FPC:prepare_fee	124 (+2%)	15,062
Token:transfer_public	⚠️ 64.3 (+49%)	31,218
FPC:pay_refund	132 (+3%)	25,260
Benchmarking:increment_balance	1,339 (-4%)	15,267
Token:_increase_public_balance	15.2 (-17%)	15,006
FPC:pay_refund_with_shielded_rebate	183 (-9%)	26,347

Public DB Access

Time to access various public DBs.

Function	time_ms
get-nullifier-index	0.165 (+8%)

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	256 leaves	512 leaves	1024 leaves
batch_insert_into_append_only_tree_16_depth_ms	10.5 (-2%)	17.5	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.8	31.7	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.609 (-3%)	0.535	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	48.8 (-3%)	76.3 (-5%)	132 (-6%)	247 (-5%)	476 (-6%)
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	95.9	159	287	543	1,055
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.498 (-3%)	0.468 (-6%)	0.452 (-6%)	0.446 (-6%)	0.444 (-6%)
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	59.8 (-4%)	112 (-6%)	184 (-6%)	355 (-5%)	700 (-6%)
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	109	207	355	691	1,363
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.505 (-5%)	0.503 (-5%)	0.486 (-5%)	0.480 (-6%)	0.479 (-6%)
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	73.2 (-5%)	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	133	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	0.520 (-5%)	N/A	N/A	N/A	N/A

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes	1 registered classes
tx_size_in_bytes	76,847	667,838

Transaction size based on fee payment method

| Metric | |
| - | |

…ckages into gj/builder-to-cli

… gj/builder-to-cli

.github/workflows/devnet-deploys.yml

alexghr

lgtm!

aztec-up/bin/aztec

aztec-up/bin/aztec-builder

… gj/builder-to-cli

AztecBot · 2024-07-15T07:59:08Z

Docs Preview

Hey there! 👋 You can check your preview at https://6694ed1e0dce1f41f87ecfed--aztec-docs-dev.netlify.app

🤖 I have created a release *beep* *boop* --- <details><summary>aztec-package: 0.46.6</summary> ## [0.46.6](aztec-package-v0.46.5...aztec-package-v0.46.6) (2024-07-15) ### Features * Modular CLI + `aztec test` ([#7426](#7426)) ([cca2a9b](cca2a9b)) </details> <details><summary>barretenberg.js: 0.46.6</summary> ## [0.46.6](barretenberg.js-v0.46.5...barretenberg.js-v0.46.6) (2024-07-15) ### Features * Modular CLI + `aztec test` ([#7426](#7426)) ([cca2a9b](cca2a9b)) </details> <details><summary>aztec-packages: 0.46.6</summary> ## [0.46.6](aztec-packages-v0.46.5...aztec-packages-v0.46.6) (2024-07-15) ### Features * Modular CLI + `aztec test` ([#7426](#7426)) ([cca2a9b](cca2a9b)) ### Bug Fixes * Aws secrets in docs CI ([#7470](#7470)) ([3b2acc7](3b2acc7)) </details> <details><summary>barretenberg: 0.46.6</summary> ## [0.46.6](barretenberg-v0.46.5...barretenberg-v0.46.6) (2024-07-15) ### Features * Modular CLI + `aztec test` ([#7426](#7426)) ([cca2a9b](cca2a9b)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- <details><summary>aztec-package: 0.46.6</summary> ## [0.46.6](AztecProtocol/aztec-packages@aztec-package-v0.46.5...aztec-package-v0.46.6) (2024-07-15) ### Features * Modular CLI + `aztec test` ([#7426](AztecProtocol/aztec-packages#7426)) ([cca2a9b](AztecProtocol/aztec-packages@cca2a9b)) </details> <details><summary>barretenberg.js: 0.46.6</summary> ## [0.46.6](AztecProtocol/aztec-packages@barretenberg.js-v0.46.5...barretenberg.js-v0.46.6) (2024-07-15) ### Features * Modular CLI + `aztec test` ([#7426](AztecProtocol/aztec-packages#7426)) ([cca2a9b](AztecProtocol/aztec-packages@cca2a9b)) </details> <details><summary>aztec-packages: 0.46.6</summary> ## [0.46.6](AztecProtocol/aztec-packages@aztec-packages-v0.46.5...aztec-packages-v0.46.6) (2024-07-15) ### Features * Modular CLI + `aztec test` ([#7426](AztecProtocol/aztec-packages#7426)) ([cca2a9b](AztecProtocol/aztec-packages@cca2a9b)) ### Bug Fixes * Aws secrets in docs CI ([#7470](AztecProtocol/aztec-packages#7470)) ([3b2acc7](AztecProtocol/aztec-packages@3b2acc7)) </details> <details><summary>barretenberg: 0.46.6</summary> ## [0.46.6](AztecProtocol/aztec-packages@barretenberg-v0.46.5...barretenberg-v0.46.6) (2024-07-15) ### Features * Modular CLI + `aztec test` ([#7426](AztecProtocol/aztec-packages#7426)) ([cca2a9b](AztecProtocol/aztec-packages@cca2a9b)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

Thunkar added 10 commits July 8, 2024 06:42

wip

56c3c7e

wip

b15c7af

separate txe in bin and exports

9fadc9e

dirty solution

29a9bdf

Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…

c329587

… gj/builder-to-cli

Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…

f63e062

… gj/builder-to-cli

wip

188b7fe

moved to start --txe model

235cbe7

fixed ports and other nits

3259d0b

cleanup

89b3587

Thunkar self-assigned this Jul 10, 2024

better wording

1eac6dd

Thunkar added 17 commits July 11, 2024 07:31

wip

a182396

started using modules

5e70043

version fixes

537c238

restored tsconfig

0a250ad

Merge branch 'master' into gj/builder-to-cli

fc7e610

naming bins

7e7dd61

Merge branch 'gj/builder-to-cli' of github.com:AztecProtocol/aztec-pa…

af88013

…ckages into gj/builder-to-cli

Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…

c844515

… gj/builder-to-cli

trying to cleanup circular deps

7969d1e

removed duplicated command and cli from earthly

554bf5b

fixed e2e

c60715d

updated lock

fbfbf50

formatting

b3810c6

yarn prepare

4b6c4f3

fixes

34c74bf

fixed formatting errors

0290479

added missing command injectors

13ac1ad

more fixes

eeb5add

Thunkar requested review from rahul-kothari, a team, alexghr and spypsy July 12, 2024 13:19

spypsy reviewed Jul 12, 2024

View reviewed changes

.github/workflows/devnet-deploys.yml Show resolved Hide resolved

alexghr approved these changes Jul 12, 2024

View reviewed changes

aztec-up/bin/aztec Show resolved Hide resolved

aztec-up/bin/aztec-builder Outdated Show resolved Hide resolved

Thunkar added 3 commits July 12, 2024 15:56

updated devnet deploy flow

d814984

Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…

ca4999c

… gj/builder-to-cli

updated docs

f832862

Thunkar marked this pull request as ready for review July 15, 2024 09:19

minor doc fixes

dcd3bff

Thunkar enabled auto-merge (squash) July 15, 2024 09:49

Thunkar merged commit cca2a9b into master Jul 15, 2024
92 checks passed

Thunkar deleted the gj/builder-to-cli branch July 15, 2024 10:34

AztecBot mentioned this pull request Jul 15, 2024

chore(master): Release 0.46.6 #7471

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: modular CLI + `aztec test` #7426

feat: modular CLI + `aztec test` #7426

Thunkar commented Jul 10, 2024 •

edited

Loading

AztecBot commented Jul 10, 2024 •

edited

Loading

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

AVM Simulation

Public DB Access

Tree insertion stats

Miscellaneous

alexghr left a comment

AztecBot commented Jul 15, 2024 •

edited

Loading

feat: modular CLI + aztec test #7426

feat: modular CLI + aztec test #7426

Conversation

Thunkar commented Jul 10, 2024 • edited Loading

AztecBot commented Jul 10, 2024 • edited Loading

Benchmark results

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

AVM Simulation

Public DB Access

Tree insertion stats

Miscellaneous

alexghr left a comment

Choose a reason for hiding this comment

AztecBot commented Jul 15, 2024 • edited Loading

Docs Preview

feat: modular CLI + `aztec test` #7426

feat: modular CLI + `aztec test` #7426

Thunkar commented Jul 10, 2024 •

edited

Loading

AztecBot commented Jul 10, 2024 •

edited

Loading

AztecBot commented Jul 15, 2024 •

edited

Loading