feat!: TXE #6985

Thunkar · 2024-06-10T10:49:07Z

This PR adds the TXE (Testing eXecution Environment, or Tricky PXE) as a minimalistic oracle resolver that mimics the behavior of the Aztec protocol in order to write smart contract tests, much like Foundry for Solidity contracts.

Together with a TestEnvironment added to aztec-nr, its aim is to make possible writing Aztec tests in pure noir resulting in much faster tests times than running e2e tests.

TXE does NOT test the protocol itself and might let you do very illegal stuff ATM, but its intent is to provide a faster way to test contract logic and contract logic only.

A 38 second Token Transfer test:

WARNING: TXE is under heavy development and we are learning how and what we want to test as we go. It's also in need of a heavy refactor, but we need this base code in order to get it in the hands of more people and understand its scope and required capabilities.

… nv/immutable

…nto gj_nv/txe

AztecBot · 2024-06-17T11:53:39Z

Benchmark results

Metrics with a significant change:

proof_construction_time_sha256_30_ms (16): 1,815 (+30%)
avm_simulation_time_ms (Token:mint_public): 64.8 (-89%)
avm_simulation_time_ms (Token:assert_minter_and_mint): 226 (+213%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric	1 threads	4 threads	16 threads	32 threads	64 threads
proof_construction_time_sha256_30_ms	11,504	3,193	⚠️ 1,815 (+30%)	1,448 (+2%)	1,466 (-1%)
proof_construction_time_sha256_100_ms	45,674 (-2%)	12,352 (-2%)	5,750 (-5%)	5,642 (-1%)	5,546 (-1%)
proof_construction_time_poseidon_hash_ms	78.0	34.0	34.0	58.0 (+2%)	87.0 (-1%)
proof_construction_time_poseidon_hash_30_ms	1,535	421	203	234 (+2%)	269
proof_construction_time_poseidon_hash_100_ms	5,811	1,599 (-1%)	741	793 (-1%)	808

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	4 txs	8 txs	16 txs
l1_rollup_calldata_size_in_bytes	1,412	1,412	1,412
l1_rollup_calldata_gas	9,452	9,460	9,440
l1_rollup_execution_gas	610,273	610,281	610,261
l2_block_processing_time_in_ms	753	1,399 (-1%)	2,679
l2_block_building_time_in_ms	25,157 (-1%)	51,781	98,511 (-1%)
l2_block_rollup_simulation_time_in_ms	25,061 (-1%)	51,490	98,160 (-1%)
l2_block_public_tx_process_time_in_ms	21,530 (-1%)	47,711	94,480 (-1%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric	3 blocks	5 blocks
node_history_sync_time_in_ms	6,952 (-2%)	9,704 (-2%)
node_database_size_in_bytes	12,177,488	16,117,840
pxe_database_size_in_bytes	16,254	26,813

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit	simulation_time_in_ms	witness_generation_time_in_ms	proving_time_in_ms	input_size_in_bytes	output_size_in_bytes	proof_size_in_bytes	num_public_inputs	size_in_gates
private-kernel-init	130	466 (-2%)	12,880 (-3%)	20,634	67,190	92,352	2,819	524,288
private-kernel-inner	393 (+1%)	1,018 (-9%)	50,337 (+1%)	94,902	67,190	92,352	2,819	2,097,152
private-kernel-tail	369 (-1%)	1,805 (+2%)	50,265	99,121	71,733	14,912	399	2,097,152
base-parity	6.26 (-4%)	1,840 (+1%)	2,811 (+3%)	128	64.0	2,208	2.00	131,072
root-parity	48.8 (-1%)	68.7 (+5%)	42,503 (+4%)	27,100	64.0	2,720	18.0	2,097,152
base-rollup	7,940	4,439 (+4%)	84,796 (-1%)	170,330	756	3,648	47.0	4,194,304
root-rollup	111 (+2%)	80.7 (+1%)	24,607 (+6%)	25,309	620	3,456	41.0	1,048,576
public-kernel-setup	718 (+1%)	3,371 (+6%)	45,126 (+1%)	116,905	93,334	125,344	3,850	2,097,152
public-kernel-app-logic	619 (+1%)	4,118 (+1%)	46,889 (+2%)	116,905	93,334	125,344	3,850	2,097,152
public-kernel-tail	1,421 (+1%)	37,153 (+6%)	188,945 (-1%)	511,910	10,014	14,912	399	8,388,608
private-kernel-reset-small	592	1,921 (-1%)	46,043 (-1%)	123,313	67,190	92,352	2,819	2,097,152
public-kernel-teardown	614 (+1%)	4,209 (+4%)	48,767 (+3%)	116,905	93,334	125,344	3,850	2,097,152
merge-rollup	28.7	N/A	N/A	16,542	756	N/A	N/A	N/A
private-kernel-tail-to-public	N/A	6,787 (-3%)	100,019 (+4%)	N/A	N/A	125,344	3,850	4,194,304
avm-circuit	N/A	N/A	4,250 (+8%)	N/A	N/A	87,734	0.00	0.00

Stats on running time collected for app circuits

Function	input_size_in_bytes	output_size_in_bytes	witness_generation_time_in_ms	proof_size_in_bytes	proving_time_in_ms	size_in_gates	num_public_inputs
ContractClassRegisterer:register	1,344	9,944	400	N/A	N/A	N/A	N/A
ContractInstanceDeployer:deploy	1,408	9,944	40.2 (+1%)	N/A	N/A	N/A	N/A
MultiCallEntrypoint:entrypoint	1,920	9,944	1,802 (+2%)	N/A	N/A	N/A	N/A
GasToken:deploy	1,376	9,944	969 (+2%)	N/A	N/A	N/A	N/A
SchnorrAccount:constructor	1,312	9,944	1,450 (+3%)	N/A	N/A	N/A	N/A
SchnorrAccount:entrypoint	2,304	9,944	2,853	16,768	56,886 (+2%)	2,097,152	457
Token:privately_mint_private_note	1,280	9,944	1,660 (+2%)	N/A	N/A	N/A	N/A
FPC:fee_entrypoint_public	1,344	9,944	357 (+2%)	16,768	11,146 (+1%)	524,288	457
Token:transfer	1,312	9,944	4,819	16,768	51,816 (+3%)	2,097,152	457
Benchmarking:create_note	1,344	9,944	1,403 (+1%)	N/A	N/A	N/A	N/A
SchnorrAccount:spend_private_authwit	1,280	9,944	76.5	N/A	N/A	N/A	N/A
Token:unshield	1,376	9,944	4,050 (-3%)	N/A	N/A	N/A	N/A
FPC:fee_entrypoint_private	1,376	9,944	4,993 (-3%)	N/A	N/A	N/A	N/A

AVM Simulation

Time to simulate various public functions in the AVM.

Function	time_ms	bytecode_size_in_bytes
GasToken:_increase_public_balance	67.8 (-1%)	13,873
GasToken:set_portal	17.0 (-2%)	3,495
Token:constructor	96.3 (+2%)	24,207
FPC:constructor	63.2 (+3%)	13,893
GasToken:mint_public	52.6 (+2%)	10,241
Token:mint_public	⚠️ 64.8 (-89%)	19,216
Token:assert_minter_and_mint	⚠️ 226 (+213%)	13,034
AuthRegistry:set_authorized	31.2 (-3%)	7,869
FPC:prepare_fee	79.6 (-2%)	15,187
Token:transfer_public	43.0 (+17%)	31,425
FPC:pay_refund	70.0 (+10%)	20,080
Benchmarking:increment_balance	2,658 (-1%)	15,465
Token:_increase_public_balance	57.0 (-3%)	15,089
FPC:pay_refund_with_shielded_rebate	51.3 (-4%)	21,167

Public DB Access

Time to access various public DBs.

Function	time_ms
get-nullifier-index	0.191 (+2%)

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	256 leaves	512 leaves	1024 leaves
batch_insert_into_append_only_tree_16_depth_ms	10.3	16.7	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.8	31.7	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.597	0.512 (+1%)	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	48.7 (+1%)	75.5	131	243 (-1%)	468
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	95.9	159	287	543	1,055
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.496	0.464	0.450	0.441 (-1%)	0.437
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	59.3	111	182 (+1%)	350 (-1%)	688
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	109	207	355	691	1,363
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.502	0.497	0.483	0.474	0.472
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	72.4 (-1%)	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	133	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	0.516 (-1%)	N/A	N/A	N/A	N/A

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes	1 registered classes
tx_size_in_bytes	85,672	670,983

Transaction size based on fee payment method

| Metric | |
| - | |

.vscode/settings.json

noir-projects/Earthfile

noir-projects/aztec-nr/aztec/src/state_vars/storage.nr

noir-projects/noir-contracts/contracts/test_contract/src/main.nr

… gj_nv/txe

LHerskind

Added a few nits, mainly around adding a few comments or deleting some stale ones and pointing to issues. Minor things really and think it is mergable without and that it is mainly nits.

noir-projects/Earthfile

noir-projects/aztec-nr/aztec/src/context/call_interfaces.nr

LHerskind · 2024-06-18T09:59:20Z

noir-projects/aztec-nr/aztec/src/state_vars/private_mutable/test.nr

+}
+
+fn in_private(env: &mut TestEnvironment) -> PrivateMutable<MockNote, &mut PrivateContext> {
+    let state_var = PrivateMutable::new(&mut env.private(), storage_slot);

    // This oracle is called for its side effects alone - it's always expected to return 0.
    OracleMock::mock("notifyCreatedNote").returns(0);


If this is something that always is returning the same, it seems like it would be useful to handle that somewhere not in the user test code if we can.

These tests are very much WIP and @nventuro is in the process of refactoring them, just used to check we can do them (but don't really know what they're going to look like)

LHerskind · 2024-06-18T10:08:30Z

noir-projects/aztec-nr/aztec/src/state_vars/public_immutable/test.nr

+use crate::test::{helpers::test_environment::TestEnvironment, mocks::mock_struct::MockStruct};
+use dep::protocol_types::traits::Serialize;
+
+global storage_slot = 7;


Is this just to have "some" value? Seems to be used just in the in_public but guess it is nicer to have it clearly named here 👍

Same as previous

LHerskind · 2024-06-18T10:09:23Z

noir-projects/aztec-nr/aztec/src/state_vars/shared_mutable/test.nr

-fn setup() -> (SharedMutable<Field, TEST_INITIAL_DELAY, &mut PublicContext>, Field) {
-    let block_number = 40;
-    let mut context = ContextBuilder::new().block_number(block_number).public();
+global TEST_INITIAL_DELAY: u32 = 30;


Pretty wild to see so long delays without it totally nuking testing time 😎

noir-projects/noir-contracts/contracts/parent_contract/src/main.nr

noir/noir-repo/aztec_macros/src/lib.rs

noir/noir-repo/aztec_macros/src/transforms/contract_interface.rs

yarn-project/txe/src/oracle/txe_oracle.ts

noir-projects/noir-contracts/contracts/token_contract/src/main.nr

… gj_nv/txe

yarn-project/package.json

noir-projects/Dockerfile

nventuro · 2024-06-18T15:11:42Z

noir-projects/aztec-nr/aztec/src/state_vars/shared_mutable/test.nr

-    let delay_change_slot = state_var.get_delay_change_storage_slot();
-    let _ = OracleMock::mock("storageRead").with_params((delay_change_slot, 1)).returns([0]).times(1);
-}
+// #[test]


No, you're 100% correct

Broken after #6985 --------- Co-authored-by: Nicolás Venturo <[email protected]>

🤖 I have created a release *beep* *boop* --- <details><summary>aztec-package: 0.44.0</summary> ## [0.44.0](aztec-package-v0.43.0...aztec-package-v0.44.0) (2024-06-26) ### Features * Add OpenTelemetry to node ([#7102](#7102)) ([6bf2b72](6bf2b72)) * Devnet deployments ([#7024](#7024)) ([fa70876](fa70876)) * Track spans ([#7129](#7129)) ([924c3f8](924c3f8)) </details> <details><summary>barretenberg.js: 0.44.0</summary> ## [0.44.0](barretenberg.js-v0.43.0...barretenberg.js-v0.44.0) (2024-06-26) ### Bug Fixes * False decryption fix ([#7066](#7066)) ([48d9df4](48d9df4)) </details> <details><summary>aztec-packages: 0.44.0</summary> ## [0.44.0](aztec-packages-v0.43.0...aztec-packages-v0.44.0) (2024-06-26) ### ⚠ BREAKING CHANGES * make note_getter return BoundedVec instead of an Option array ([#7050](#7050)) * TXE ([#6985](#6985)) ### Features * Add macro impls for events ([#7081](#7081)) ([c13dd9f](c13dd9f)) * Add OpenTelemetry to node ([#7102](#7102)) ([6bf2b72](6bf2b72)) * Added prove_output_all flow for honk ([#6869](#6869)) ([7bd7c66](7bd7c66)) * **avm:** Add ECC ops to avm_proving_test ([#7058](#7058)) ([7f62a90](7f62a90)) * **avm:** Cpp msm changes ([#7056](#7056)) ([f9c8f20](f9c8f20)) * **avm:** Include bb-pilcom in monorepo ([#7098](#7098)) ([0442158](0442158)) * Constrain event encryption and unify note and event emit api ([#7171](#7171)) ([5c3772f](5c3772f)), closes [#7160](#7160) * Conventional lookups using log-deriv ([#7020](#7020)) ([6f1212f](6f1212f)) * Devnet deployments ([#7024](#7024)) ([fa70876](fa70876)) * Do not discard logs on revert since the kernel has pruned revertible logs. ([#7076](#7076)) ([366fb21](366fb21)), closes [#4712](#4712) * **docs:** Publish PDF of protocol specs + remove links to pages in item lists in protocol specs ([#6684](#6684)) ([367e3cf](367e3cf)) * Enable merge recursive verifier in Goblin recursive verifier ([#7182](#7182)) ([9b4f56c](9b4f56c)) * Flamegraph helper script ([#7077](#7077)) ([8630c8f](8630c8f)) * Full test skips public simulation ([#7186](#7186)) ([4c1997f](4c1997f)) * Make note_getter return BoundedVec instead of an Option array ([#7050](#7050)) ([f9ac0fc](f9ac0fc)) * **p2p:** More comprehensive peer management, dial retries, persistence fix ([#6953](#6953)) ([cdd1cbd](cdd1cbd)) * Private authwit with static call ([#7073](#7073)) ([9c52d47](9c52d47)) * Several updates in SMT verification module ([#7105](#7105)) ([41b21f1](41b21f1)) * Shplonk revival in ECCVM ([#7164](#7164)) ([34eb5a0](34eb5a0)) * Throwing errors in `BufferReader` when out of bounds ([#7149](#7149)) ([bf4a986](bf4a986)) * Track spans ([#7129](#7129)) ([924c3f8](924c3f8)) * TXE ([#6985](#6985)) ([109624f](109624f)) * TXE 2: Electric boogaloo ([#7154](#7154)) ([bb38246](bb38246)) ### Bug Fixes * **avm:** Fix unencryptedlog c++ deser ([#7194](#7194)) ([89a99af](89a99af)) * **avm:** Re-enable ext call test ([#7147](#7147)) ([33ccf1b](33ccf1b)) * **avm:** Reenable tag error sload ([#7153](#7153)) ([fd92d46](fd92d46)) * **avm:** Update codegen ([#7178](#7178)) ([1d29708](1d29708)) * Bug fixing bench prover test ([#7135](#7135)) ([13678be](13678be)), closes [#7080](#7080) * **ci:** Don't run npm_deploy l1-contracts ([#7187](#7187)) ([80d26d8](80d26d8)) * **ci:** Move osxcross from build image ([#7151](#7151)) ([7746363](7746363)) * Enable log filtering with the DEBUG variable ([#7150](#7150)) ([33798b6](33798b6)) * Export event selector and replace function selector with event selector where appropriate ([#7095](#7095)) ([fcc15fa](fcc15fa)), closes [#7089](#7089) * False decryption fix ([#7066](#7066)) ([48d9df4](48d9df4)) * Fix bug for a unit test in full proving mode repated to MSM ([#7104](#7104)) ([e37809b](e37809b)) ### Miscellaneous * `destroy_note(...)` optimization ([#7103](#7103)) ([0770011](0770011)) * Add avm team as codeowners to more repo files ([#7196](#7196)) ([9be0ad6](9be0ad6)) * **avm:** Remove avm prefix from pil and executor ([#7099](#7099)) ([b502fcd](b502fcd)) * **avm:** Renamings and comments ([#7128](#7128)) ([ed2f98e](ed2f98e)) * **avm:** Separate some fixed tables ([#7163](#7163)) ([1d4a9a2](1d4a9a2)) * **ci:** Add new e2e base target ([#7179](#7179)) ([26fc599](26fc599)) * Create workflow for full AVM tests ([#7051](#7051)) ([a0b9c4b](a0b9c4b)), closes [#6643](#6643) * **docs:** Fix migration notes ([#7195](#7195)) ([88efda0](88efda0)) * **docs:** Moving tutorials and quick starts around, spinning off codespaces page ([#6777](#6777)) ([1542fa6](1542fa6)) * Fix migration notes ([#7133](#7133)) ([14917d3](14917d3)) * Fix noir-projects dockerfile for CircleCI ([#7093](#7093)) ([52ce25d](52ce25d)) * Increase the timeout of the runner for full AVM workflow to 70 minutes ([#7183](#7183)) ([9aabc32](9aabc32)) * Indirects and read/write slices ([#7082](#7082)) ([d5e80ee](d5e80ee)) * Note hashes cleanup + optimization ([#7132](#7132)) ([edd6d3f](edd6d3f)) * Note hashing gate optimizations ([#7130](#7130)) ([81a2580](81a2580)) * **powdr:** Update to latest and add logging ([#7152](#7152)) ([f500f2e](f500f2e)) * Reads the return data ([#6669](#6669)) ([ef85542](ef85542)) * Refactor AVM simulator's side-effect tracing ([#7091](#7091)) ([9495413](9495413)) * Remove stray files ([#7158](#7158)) ([29398de](29398de)) * Remove unneeded public input folding ([#7094](#7094)) ([c30dc38](c30dc38)) * Replace relative paths to noir-protocol-circuits ([f7e4392](f7e4392)) * Replace relative paths to noir-protocol-circuits ([886f7b1](886f7b1)) * Replace relative paths to noir-protocol-circuits ([b1081f8](b1081f8)) * Replace relative paths to noir-protocol-circuits ([c0989eb](c0989eb)) * Replace relative paths to noir-protocol-circuits ([525bbe7](525bbe7)) * Replace relative paths to noir-protocol-circuits ([67bcd82](67bcd82)) * Take the PCS out of Zeromorph and refactor tests ([#7078](#7078)) ([e192678](e192678)) * Track avm proving time ([#7084](#7084)) ([59df722](59df722)) * Ultra flavor cleanup ([#7070](#7070)) ([77761c6](77761c6)) </details> <details><summary>barretenberg: 0.44.0</summary> ## [0.44.0](barretenberg-v0.43.0...barretenberg-v0.44.0) (2024-06-26) ### Features * Added prove_output_all flow for honk ([#6869](#6869)) ([7bd7c66](7bd7c66)) * **avm:** Add ECC ops to avm_proving_test ([#7058](#7058)) ([7f62a90](7f62a90)) * **avm:** Cpp msm changes ([#7056](#7056)) ([f9c8f20](f9c8f20)) * **avm:** Include bb-pilcom in monorepo ([#7098](#7098)) ([0442158](0442158)) * Conventional lookups using log-deriv ([#7020](#7020)) ([6f1212f](6f1212f)) * Enable merge recursive verifier in Goblin recursive verifier ([#7182](#7182)) ([9b4f56c](9b4f56c)) * Several updates in SMT verification module ([#7105](#7105)) ([41b21f1](41b21f1)) * Shplonk revival in ECCVM ([#7164](#7164)) ([34eb5a0](34eb5a0)) ### Bug Fixes * **avm:** Fix unencryptedlog c++ deser ([#7194](#7194)) ([89a99af](89a99af)) * **avm:** Re-enable ext call test ([#7147](#7147)) ([33ccf1b](33ccf1b)) * **avm:** Reenable tag error sload ([#7153](#7153)) ([fd92d46](fd92d46)) * **avm:** Update codegen ([#7178](#7178)) ([1d29708](1d29708)) * Bug fixing bench prover test ([#7135](#7135)) ([13678be](13678be)), closes [#7080](#7080) * Fix bug for a unit test in full proving mode repated to MSM ([#7104](#7104)) ([e37809b](e37809b)) ### Miscellaneous * **avm:** Remove avm prefix from pil and executor ([#7099](#7099)) ([b502fcd](b502fcd)) * **avm:** Renamings and comments ([#7128](#7128)) ([ed2f98e](ed2f98e)) * **avm:** Separate some fixed tables ([#7163](#7163)) ([1d4a9a2](1d4a9a2)) * Create workflow for full AVM tests ([#7051](#7051)) ([a0b9c4b](a0b9c4b)), closes [#6643](#6643) * Indirects and read/write slices ([#7082](#7082)) ([d5e80ee](d5e80ee)) * Reads the return data ([#6669](#6669)) ([ef85542](ef85542)) * Remove unneeded public input folding ([#7094](#7094)) ([c30dc38](c30dc38)) * Take the PCS out of Zeromorph and refactor tests ([#7078](#7078)) ([e192678](e192678)) * Ultra flavor cleanup ([#7070](#7070)) ([77761c6](77761c6)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- <details><summary>aztec-package: 0.44.0</summary> ## [0.44.0](AztecProtocol/aztec-packages@aztec-package-v0.43.0...aztec-package-v0.44.0) (2024-06-26) ### Features * Add OpenTelemetry to node ([#7102](AztecProtocol/aztec-packages#7102)) ([6bf2b72](AztecProtocol/aztec-packages@6bf2b72)) * Devnet deployments ([#7024](AztecProtocol/aztec-packages#7024)) ([fa70876](AztecProtocol/aztec-packages@fa70876)) * Track spans ([#7129](AztecProtocol/aztec-packages#7129)) ([924c3f8](AztecProtocol/aztec-packages@924c3f8)) </details> <details><summary>barretenberg.js: 0.44.0</summary> ## [0.44.0](AztecProtocol/aztec-packages@barretenberg.js-v0.43.0...barretenberg.js-v0.44.0) (2024-06-26) ### Bug Fixes * False decryption fix ([#7066](AztecProtocol/aztec-packages#7066)) ([48d9df4](AztecProtocol/aztec-packages@48d9df4)) </details> <details><summary>aztec-packages: 0.44.0</summary> ## [0.44.0](AztecProtocol/aztec-packages@aztec-packages-v0.43.0...aztec-packages-v0.44.0) (2024-06-26) ### ⚠ BREAKING CHANGES * make note_getter return BoundedVec instead of an Option array ([#7050](AztecProtocol/aztec-packages#7050)) * TXE ([#6985](AztecProtocol/aztec-packages#6985)) ### Features * Add macro impls for events ([#7081](AztecProtocol/aztec-packages#7081)) ([c13dd9f](AztecProtocol/aztec-packages@c13dd9f)) * Add OpenTelemetry to node ([#7102](AztecProtocol/aztec-packages#7102)) ([6bf2b72](AztecProtocol/aztec-packages@6bf2b72)) * Added prove_output_all flow for honk ([#6869](AztecProtocol/aztec-packages#6869)) ([7bd7c66](AztecProtocol/aztec-packages@7bd7c66)) * **avm:** Add ECC ops to avm_proving_test ([#7058](AztecProtocol/aztec-packages#7058)) ([7f62a90](AztecProtocol/aztec-packages@7f62a90)) * **avm:** Cpp msm changes ([#7056](AztecProtocol/aztec-packages#7056)) ([f9c8f20](AztecProtocol/aztec-packages@f9c8f20)) * **avm:** Include bb-pilcom in monorepo ([#7098](AztecProtocol/aztec-packages#7098)) ([0442158](AztecProtocol/aztec-packages@0442158)) * Constrain event encryption and unify note and event emit api ([#7171](AztecProtocol/aztec-packages#7171)) ([5c3772f](AztecProtocol/aztec-packages@5c3772f)), closes [#7160](AztecProtocol/aztec-packages#7160) * Conventional lookups using log-deriv ([#7020](AztecProtocol/aztec-packages#7020)) ([6f1212f](AztecProtocol/aztec-packages@6f1212f)) * Devnet deployments ([#7024](AztecProtocol/aztec-packages#7024)) ([fa70876](AztecProtocol/aztec-packages@fa70876)) * Do not discard logs on revert since the kernel has pruned revertible logs. ([#7076](AztecProtocol/aztec-packages#7076)) ([366fb21](AztecProtocol/aztec-packages@366fb21)), closes [#4712](AztecProtocol/aztec-packages#4712) * **docs:** Publish PDF of protocol specs + remove links to pages in item lists in protocol specs ([#6684](AztecProtocol/aztec-packages#6684)) ([367e3cf](AztecProtocol/aztec-packages@367e3cf)) * Enable merge recursive verifier in Goblin recursive verifier ([#7182](AztecProtocol/aztec-packages#7182)) ([9b4f56c](AztecProtocol/aztec-packages@9b4f56c)) * Flamegraph helper script ([#7077](AztecProtocol/aztec-packages#7077)) ([8630c8f](AztecProtocol/aztec-packages@8630c8f)) * Full test skips public simulation ([#7186](AztecProtocol/aztec-packages#7186)) ([4c1997f](AztecProtocol/aztec-packages@4c1997f)) * Make note_getter return BoundedVec instead of an Option array ([#7050](AztecProtocol/aztec-packages#7050)) ([f9ac0fc](AztecProtocol/aztec-packages@f9ac0fc)) * **p2p:** More comprehensive peer management, dial retries, persistence fix ([#6953](AztecProtocol/aztec-packages#6953)) ([cdd1cbd](AztecProtocol/aztec-packages@cdd1cbd)) * Private authwit with static call ([#7073](AztecProtocol/aztec-packages#7073)) ([9c52d47](AztecProtocol/aztec-packages@9c52d47)) * Several updates in SMT verification module ([#7105](AztecProtocol/aztec-packages#7105)) ([41b21f1](AztecProtocol/aztec-packages@41b21f1)) * Shplonk revival in ECCVM ([#7164](AztecProtocol/aztec-packages#7164)) ([34eb5a0](AztecProtocol/aztec-packages@34eb5a0)) * Throwing errors in `BufferReader` when out of bounds ([#7149](AztecProtocol/aztec-packages#7149)) ([bf4a986](AztecProtocol/aztec-packages@bf4a986)) * Track spans ([#7129](AztecProtocol/aztec-packages#7129)) ([924c3f8](AztecProtocol/aztec-packages@924c3f8)) * TXE ([#6985](AztecProtocol/aztec-packages#6985)) ([109624f](AztecProtocol/aztec-packages@109624f)) * TXE 2: Electric boogaloo ([#7154](AztecProtocol/aztec-packages#7154)) ([bb38246](AztecProtocol/aztec-packages@bb38246)) ### Bug Fixes * **avm:** Fix unencryptedlog c++ deser ([#7194](AztecProtocol/aztec-packages#7194)) ([89a99af](AztecProtocol/aztec-packages@89a99af)) * **avm:** Re-enable ext call test ([#7147](AztecProtocol/aztec-packages#7147)) ([33ccf1b](AztecProtocol/aztec-packages@33ccf1b)) * **avm:** Reenable tag error sload ([#7153](AztecProtocol/aztec-packages#7153)) ([fd92d46](AztecProtocol/aztec-packages@fd92d46)) * **avm:** Update codegen ([#7178](AztecProtocol/aztec-packages#7178)) ([1d29708](AztecProtocol/aztec-packages@1d29708)) * Bug fixing bench prover test ([#7135](AztecProtocol/aztec-packages#7135)) ([13678be](AztecProtocol/aztec-packages@13678be)), closes [#7080](AztecProtocol/aztec-packages#7080) * **ci:** Don't run npm_deploy l1-contracts ([#7187](AztecProtocol/aztec-packages#7187)) ([80d26d8](AztecProtocol/aztec-packages@80d26d8)) * **ci:** Move osxcross from build image ([#7151](AztecProtocol/aztec-packages#7151)) ([7746363](AztecProtocol/aztec-packages@7746363)) * Enable log filtering with the DEBUG variable ([#7150](AztecProtocol/aztec-packages#7150)) ([33798b6](AztecProtocol/aztec-packages@33798b6)) * Export event selector and replace function selector with event selector where appropriate ([#7095](AztecProtocol/aztec-packages#7095)) ([fcc15fa](AztecProtocol/aztec-packages@fcc15fa)), closes [#7089](AztecProtocol/aztec-packages#7089) * False decryption fix ([#7066](AztecProtocol/aztec-packages#7066)) ([48d9df4](AztecProtocol/aztec-packages@48d9df4)) * Fix bug for a unit test in full proving mode repated to MSM ([#7104](AztecProtocol/aztec-packages#7104)) ([e37809b](AztecProtocol/aztec-packages@e37809b)) ### Miscellaneous * `destroy_note(...)` optimization ([#7103](AztecProtocol/aztec-packages#7103)) ([0770011](AztecProtocol/aztec-packages@0770011)) * Add avm team as codeowners to more repo files ([#7196](AztecProtocol/aztec-packages#7196)) ([9be0ad6](AztecProtocol/aztec-packages@9be0ad6)) * **avm:** Remove avm prefix from pil and executor ([#7099](AztecProtocol/aztec-packages#7099)) ([b502fcd](AztecProtocol/aztec-packages@b502fcd)) * **avm:** Renamings and comments ([#7128](AztecProtocol/aztec-packages#7128)) ([ed2f98e](AztecProtocol/aztec-packages@ed2f98e)) * **avm:** Separate some fixed tables ([#7163](AztecProtocol/aztec-packages#7163)) ([1d4a9a2](AztecProtocol/aztec-packages@1d4a9a2)) * **ci:** Add new e2e base target ([#7179](AztecProtocol/aztec-packages#7179)) ([26fc599](AztecProtocol/aztec-packages@26fc599)) * Create workflow for full AVM tests ([#7051](AztecProtocol/aztec-packages#7051)) ([a0b9c4b](AztecProtocol/aztec-packages@a0b9c4b)), closes [#6643](AztecProtocol/aztec-packages#6643) * **docs:** Fix migration notes ([#7195](AztecProtocol/aztec-packages#7195)) ([88efda0](AztecProtocol/aztec-packages@88efda0)) * **docs:** Moving tutorials and quick starts around, spinning off codespaces page ([#6777](AztecProtocol/aztec-packages#6777)) ([1542fa6](AztecProtocol/aztec-packages@1542fa6)) * Fix migration notes ([#7133](AztecProtocol/aztec-packages#7133)) ([14917d3](AztecProtocol/aztec-packages@14917d3)) * Fix noir-projects dockerfile for CircleCI ([#7093](AztecProtocol/aztec-packages#7093)) ([52ce25d](AztecProtocol/aztec-packages@52ce25d)) * Increase the timeout of the runner for full AVM workflow to 70 minutes ([#7183](AztecProtocol/aztec-packages#7183)) ([9aabc32](AztecProtocol/aztec-packages@9aabc32)) * Indirects and read/write slices ([#7082](AztecProtocol/aztec-packages#7082)) ([d5e80ee](AztecProtocol/aztec-packages@d5e80ee)) * Note hashes cleanup + optimization ([#7132](AztecProtocol/aztec-packages#7132)) ([edd6d3f](AztecProtocol/aztec-packages@edd6d3f)) * Note hashing gate optimizations ([#7130](AztecProtocol/aztec-packages#7130)) ([81a2580](AztecProtocol/aztec-packages@81a2580)) * **powdr:** Update to latest and add logging ([#7152](AztecProtocol/aztec-packages#7152)) ([f500f2e](AztecProtocol/aztec-packages@f500f2e)) * Reads the return data ([#6669](AztecProtocol/aztec-packages#6669)) ([ef85542](AztecProtocol/aztec-packages@ef85542)) * Refactor AVM simulator's side-effect tracing ([#7091](AztecProtocol/aztec-packages#7091)) ([9495413](AztecProtocol/aztec-packages@9495413)) * Remove stray files ([#7158](AztecProtocol/aztec-packages#7158)) ([29398de](AztecProtocol/aztec-packages@29398de)) * Remove unneeded public input folding ([#7094](AztecProtocol/aztec-packages#7094)) ([c30dc38](AztecProtocol/aztec-packages@c30dc38)) * Replace relative paths to noir-protocol-circuits ([f7e4392](AztecProtocol/aztec-packages@f7e4392)) * Replace relative paths to noir-protocol-circuits ([886f7b1](AztecProtocol/aztec-packages@886f7b1)) * Replace relative paths to noir-protocol-circuits ([b1081f8](AztecProtocol/aztec-packages@b1081f8)) * Replace relative paths to noir-protocol-circuits ([c0989eb](AztecProtocol/aztec-packages@c0989eb)) * Replace relative paths to noir-protocol-circuits ([525bbe7](AztecProtocol/aztec-packages@525bbe7)) * Replace relative paths to noir-protocol-circuits ([67bcd82](AztecProtocol/aztec-packages@67bcd82)) * Take the PCS out of Zeromorph and refactor tests ([#7078](AztecProtocol/aztec-packages#7078)) ([e192678](AztecProtocol/aztec-packages@e192678)) * Track avm proving time ([#7084](AztecProtocol/aztec-packages#7084)) ([59df722](AztecProtocol/aztec-packages@59df722)) * Ultra flavor cleanup ([#7070](AztecProtocol/aztec-packages#7070)) ([77761c6](AztecProtocol/aztec-packages@77761c6)) </details> <details><summary>barretenberg: 0.44.0</summary> ## [0.44.0](AztecProtocol/aztec-packages@barretenberg-v0.43.0...barretenberg-v0.44.0) (2024-06-26) ### Features * Added prove_output_all flow for honk ([#6869](AztecProtocol/aztec-packages#6869)) ([7bd7c66](AztecProtocol/aztec-packages@7bd7c66)) * **avm:** Add ECC ops to avm_proving_test ([#7058](AztecProtocol/aztec-packages#7058)) ([7f62a90](AztecProtocol/aztec-packages@7f62a90)) * **avm:** Cpp msm changes ([#7056](AztecProtocol/aztec-packages#7056)) ([f9c8f20](AztecProtocol/aztec-packages@f9c8f20)) * **avm:** Include bb-pilcom in monorepo ([#7098](AztecProtocol/aztec-packages#7098)) ([0442158](AztecProtocol/aztec-packages@0442158)) * Conventional lookups using log-deriv ([#7020](AztecProtocol/aztec-packages#7020)) ([6f1212f](AztecProtocol/aztec-packages@6f1212f)) * Enable merge recursive verifier in Goblin recursive verifier ([#7182](AztecProtocol/aztec-packages#7182)) ([9b4f56c](AztecProtocol/aztec-packages@9b4f56c)) * Several updates in SMT verification module ([#7105](AztecProtocol/aztec-packages#7105)) ([41b21f1](AztecProtocol/aztec-packages@41b21f1)) * Shplonk revival in ECCVM ([#7164](AztecProtocol/aztec-packages#7164)) ([34eb5a0](AztecProtocol/aztec-packages@34eb5a0)) ### Bug Fixes * **avm:** Fix unencryptedlog c++ deser ([#7194](AztecProtocol/aztec-packages#7194)) ([89a99af](AztecProtocol/aztec-packages@89a99af)) * **avm:** Re-enable ext call test ([#7147](AztecProtocol/aztec-packages#7147)) ([33ccf1b](AztecProtocol/aztec-packages@33ccf1b)) * **avm:** Reenable tag error sload ([#7153](AztecProtocol/aztec-packages#7153)) ([fd92d46](AztecProtocol/aztec-packages@fd92d46)) * **avm:** Update codegen ([#7178](AztecProtocol/aztec-packages#7178)) ([1d29708](AztecProtocol/aztec-packages@1d29708)) * Bug fixing bench prover test ([#7135](AztecProtocol/aztec-packages#7135)) ([13678be](AztecProtocol/aztec-packages@13678be)), closes [#7080](AztecProtocol/aztec-packages#7080) * Fix bug for a unit test in full proving mode repated to MSM ([#7104](AztecProtocol/aztec-packages#7104)) ([e37809b](AztecProtocol/aztec-packages@e37809b)) ### Miscellaneous * **avm:** Remove avm prefix from pil and executor ([#7099](AztecProtocol/aztec-packages#7099)) ([b502fcd](AztecProtocol/aztec-packages@b502fcd)) * **avm:** Renamings and comments ([#7128](AztecProtocol/aztec-packages#7128)) ([ed2f98e](AztecProtocol/aztec-packages@ed2f98e)) * **avm:** Separate some fixed tables ([#7163](AztecProtocol/aztec-packages#7163)) ([1d4a9a2](AztecProtocol/aztec-packages@1d4a9a2)) * Create workflow for full AVM tests ([#7051](AztecProtocol/aztec-packages#7051)) ([a0b9c4b](AztecProtocol/aztec-packages@a0b9c4b)), closes [#6643](AztecProtocol/aztec-packages#6643) * Indirects and read/write slices ([#7082](AztecProtocol/aztec-packages#7082)) ([d5e80ee](AztecProtocol/aztec-packages@d5e80ee)) * Reads the return data ([#6669](AztecProtocol/aztec-packages#6669)) ([ef85542](AztecProtocol/aztec-packages@ef85542)) * Remove unneeded public input folding ([#7094](AztecProtocol/aztec-packages#7094)) ([c30dc38](AztecProtocol/aztec-packages@c30dc38)) * Take the PCS out of Zeromorph and refactor tests ([#7078](AztecProtocol/aztec-packages#7078)) ([e192678](AztecProtocol/aztec-packages@e192678)) * Ultra flavor cleanup ([#7070](AztecProtocol/aztec-packages#7070)) ([77761c6](AztecProtocol/aztec-packages@77761c6)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

This PR changes public write functions so that they not only store the new delay and/or value, but also a hash of the combined serialization of them. This lets us produce smaller proofs in private, since we only need to prove inclusion of the hash and not each of the individual 4 values. This will result in even larger savings once we do #5491. The only annoying bit is that producing the unconstrained hint so that we can contrain the hash and prove inclusion involves performing reads, and hence computation of storage slots. But because we cannot pass mutable references to unconstrained functions, we cannot have methods for the `&mut PrivateContext` impl, resulting in a bit of a hack to create a dummy state variable that I can call methods on. This is all going to go away regardless once #5492 is done. I also restored the tests deleted in #6985, updating the existing ones and adding new cases. --------- Co-authored-by: LHerskind <[email protected]>

nventuro and others added 30 commits May 30, 2024 20:45

Adapt public immutable

39e88bf

Add comments to private imnmutable

de4dfef

Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…

584250d

… nv/immutable

TXE

8c5bdf1

improvements

3fbc3ee

fixed tsconfig

58c4c82

public data tree witness

84af5e8

return null

aab9959

more oracle calls

c94d32e

more oracles, timetravel

5d857ae

fix

717a16a

Sketch nr tests for txe testing

58e7bfa

fix

d543edf

Begin test refactor

92b9a48

fix

c6fdef6

fixes

b29b3a4

fix

6628ab0

private context inputs

460a1d1

return struct

069a09a

fix

7edc719

configurable private inputs

f2820a7

???

cbe1564

???

2250b98

fixes

45434f9

git add .

c180bdb

fixes and moar

a34bf85

note handling

fdbeaf8

Samples

12453ea

Add address fixes

fdd5cf1

almost there

ff3fa71

Merge branch 'gj_nv/txe' of github.com:AztecProtocol/aztec-packages i…

83c16e5

…nto gj_nv/txe

Thunkar added 2 commits June 17, 2024 14:24

working CI

f564d92

better dockerfile

083b1f8

Thunkar commented Jun 17, 2024

View reviewed changes

.vscode/settings.json Show resolved Hide resolved

Thunkar commented Jun 17, 2024

View reviewed changes

noir-projects/Earthfile Show resolved Hide resolved

Thunkar commented Jun 17, 2024

View reviewed changes

noir-projects/aztec-nr/aztec/src/state_vars/storage.nr Show resolved Hide resolved

Thunkar commented Jun 17, 2024

View reviewed changes

noir-projects/noir-contracts/contracts/test_contract/src/main.nr Show resolved Hide resolved

Thunkar marked this pull request as ready for review June 17, 2024 16:08

LHerskind self-requested a review June 17, 2024 19:20

Thunkar added 3 commits June 18, 2024 07:30

Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…

2155e79

… gj_nv/txe

Merge branch 'master' into gj_nv/txe

c583065

nargo fmt

ab5e0e9

LHerskind approved these changes Jun 18, 2024

View reviewed changes

Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…

44669d9

… gj_nv/txe

rahul-kothari reviewed Jun 18, 2024

View reviewed changes

yarn-project/package.json Show resolved Hide resolved

comments from PR

39b5f76

nventuro reviewed Jun 18, 2024

View reviewed changes

Thunkar added 2 commits June 18, 2024 16:04

kill txe even if nargo test fails

9e0ac91

Merge branch 'master' into gj_nv/txe

fa744a5

Thunkar merged commit 109624f into master Jun 18, 2024
89 checks passed

Thunkar deleted the gj_nv/txe branch June 18, 2024 16:57

AztecBot mentioned this pull request Jun 18, 2024

chore(master): Release 0.44.0 #7088

Merged

Thunkar mentioned this pull request Jun 18, 2024

chore: Fix noir-projects dockerfile for CircleCI #7093

Merged

Thunkar added a commit that referenced this pull request Jun 19, 2024

chore: Fix noir-projects dockerfile for CircleCI (#7093)

52ce25d

Broken after #6985 --------- Co-authored-by: Nicolás Venturo <[email protected]>

nventuro mentioned this pull request Jun 24, 2024

feat: store shared mutable hash #7169

Merged

AztecBot mentioned this pull request Jun 26, 2024

feat: Sync from aztec-packages noir-lang/noir#5340

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: TXE #6985

feat!: TXE #6985

Thunkar commented Jun 10, 2024 •

edited

Loading

AztecBot commented Jun 17, 2024 •

edited

Loading

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

AVM Simulation

Public DB Access

Tree insertion stats

Miscellaneous

LHerskind left a comment

LHerskind Jun 18, 2024

Thunkar Jun 18, 2024

LHerskind Jun 18, 2024

Thunkar Jun 18, 2024

LHerskind Jun 18, 2024

nventuro Jun 18, 2024

feat!: TXE #6985

feat!: TXE #6985

Conversation

Thunkar commented Jun 10, 2024 • edited Loading

AztecBot commented Jun 17, 2024 • edited Loading

Benchmark results

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

AVM Simulation

Public DB Access

Tree insertion stats

Miscellaneous

LHerskind left a comment

Choose a reason for hiding this comment

LHerskind Jun 18, 2024

Choose a reason for hiding this comment

Thunkar Jun 18, 2024

Choose a reason for hiding this comment

LHerskind Jun 18, 2024

Choose a reason for hiding this comment

Thunkar Jun 18, 2024

Choose a reason for hiding this comment

LHerskind Jun 18, 2024

Choose a reason for hiding this comment

nventuro Jun 18, 2024

Choose a reason for hiding this comment

Thunkar commented Jun 10, 2024 •

edited

Loading

AztecBot commented Jun 17, 2024 •

edited

Loading