Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: TXE fixes to avm opcodes and missing oracles, forced ci failure #7252

Merged
merged 19 commits into from
Jul 2, 2024

Conversation

Thunkar
Copy link
Contributor

@Thunkar Thunkar commented Jul 1, 2024

Makes CI fail in case TXE test fail. Noticed they were broken, but master was still green and removed some unnecessary commands in the docker build.

Also reverted some of the changes from #7237 since we need the length of the data we're about to read in avmOpcodeStorageRead (the oracle cannot obtain this information from the assigned output variable.

@Thunkar Thunkar self-assigned this Jul 1, 2024
@Thunkar Thunkar requested a review from nventuro July 1, 2024 11:51
@AztecBot
Copy link
Collaborator

AztecBot commented Jul 1, 2024

Benchmark results

Metrics with a significant change:

  • avm_simulation_time_ms (Token:mint_public): 585 (+709%)
  • avm_simulation_time_ms (Token:assert_minter_and_mint): 65.3 (-70%)
  • protocol_circuit_witness_generation_time_in_ms (private-kernel-reset-small): 2,388 (+22%)
Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric 1 threads 4 threads 16 threads 32 threads 64 threads
proof_construction_time_sha256_30_ms 11,702 (-3%) 3,142 (-3%) 1,724 (-7%) 1,548 (-4%) 1,564
proof_construction_time_sha256_100_ms 43,709 (-3%) 11,720 (-2%) 5,424 (-2%) 5,404 (-1%) 5,346 (-2%)
proof_construction_time_poseidon_hash_ms 78.0 (-2%) 34.0 (-3%) 34.0 (-3%) 58.0 (-2%) 88.0
proof_construction_time_poseidon_hash_30_ms 1,517 (-3%) 414 (-2%) 200 (-1%) 224 (-3%) 266 (-2%)
proof_construction_time_poseidon_hash_100_ms 5,731 (-3%) 1,558 (-2%) 721 (-2%) 770 (-1%) 795 (-2%)

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 4 txs 8 txs 16 txs
l1_rollup_calldata_size_in_bytes 1,412 1,412 1,412
l1_rollup_calldata_gas 9,464 9,468 9,464
l1_rollup_execution_gas 611,203 611,358 611,505
l2_block_processing_time_in_ms 765 (-2%) 1,413 2,709 (-3%)
l2_block_building_time_in_ms 24,981 (-5%) 49,435 98,166 (-5%)
l2_block_rollup_simulation_time_in_ms 24,980 (-5%) 49,435 98,166 (-5%)
l2_block_public_tx_process_time_in_ms 21,477 (-5%) 45,742 94,455 (-5%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric 3 blocks 5 blocks
node_history_sync_time_in_ms 7,032 9,834 (-1%)
node_database_size_in_bytes 12,247,120 16,113,744
pxe_database_size_in_bytes 16,254 26,813

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit simulation_time_in_ms witness_generation_time_in_ms proving_time_in_ms input_size_in_bytes output_size_in_bytes proof_size_in_bytes num_public_inputs size_in_gates
private-kernel-init 115 (-1%) 458 12,993 (-2%) 20,634 67,190 92,352 2,819 524,288
private-kernel-inner 359 (-1%) 908 (-15%) 49,260 (+1%) 94,902 67,190 92,352 2,819 2,097,152
private-kernel-tail 302 (-1%) 1,725 (-2%) 49,208 (+1%) 99,121 71,733 14,912 399 2,097,152
base-parity 6.46 (+3%) 1,769 (-2%) 2,752 (-3%) 128 64.0 2,208 2.00 131,072
root-parity 49.3 (-2%) 81.5 (+9%) 40,751 (-2%) 27,100 64.0 2,720 18.0 2,097,152
base-rollup 7,723 (-2%) 4,919 (-1%) 84,032 (-1%) 170,330 728 3,648 47.0 4,194,304
root-rollup 110 (-2%) 85.7 (-7%) 23,515 25,253 620 3,456 41.0 1,048,576
public-kernel-setup 640 3,728 (-2%) 44,907 (-1%) 116,905 93,334 125,344 3,850 2,097,152
public-kernel-app-logic 588 (-2%) 4,798 45,326 (-7%) 116,905 93,334 125,344 3,850 2,097,152
public-kernel-tail 1,387 (-2%) 40,280 (+3%) 192,603 (+4%) 511,910 10,014 14,912 399 8,388,608
private-kernel-reset-small 553 (-1%) ⚠️ 2,388 (+22%) 47,372 (-1%) 123,313 67,190 92,352 2,819 2,097,152
public-kernel-teardown 581 (+1%) 4,785 (-2%) 47,291 (-2%) 116,905 93,334 125,344 3,850 2,097,152
merge-rollup 30.0 N/A N/A 16,486 728 N/A N/A N/A
private-kernel-tail-to-public N/A 7,204 (+4%) 97,511 (+2%) N/A N/A 125,344 3,850 4,194,304

Stats on running time collected for app circuits

Function input_size_in_bytes output_size_in_bytes witness_generation_time_in_ms proof_size_in_bytes proving_time_in_ms size_in_gates num_public_inputs
ContractClassRegisterer:register 1,344 9,944 404 (-3%) N/A N/A N/A N/A
ContractInstanceDeployer:deploy 1,408 9,944 39.2 (-2%) N/A N/A N/A N/A
MultiCallEntrypoint:entrypoint 1,920 9,944 1,293 (-2%) N/A N/A N/A N/A
GasToken:deploy 1,376 9,944 950 (-1%) N/A N/A N/A N/A
SchnorrAccount:constructor 1,312 9,944 488 (-2%) N/A N/A N/A N/A
SchnorrAccount:entrypoint 2,304 9,944 1,870 (-1%) 16,768 56,338 (+1%) 2,097,152 457
Token:privately_mint_private_note 1,280 9,944 636 (-1%) N/A N/A N/A N/A
FPC:fee_entrypoint_public 1,344 9,944 304 (-1%) 16,768 11,772 (+2%) 524,288 457
Token:transfer 1,312 9,944 2,905 16,768 23,585 (+4%) 1,048,576 457
AuthRegistry:set_authorized (avm) 20,954 N/A N/A 94,336 1,344 (-6%) N/A N/A
FPC:prepare_fee (avm) 28,396 N/A N/A 94,400 2,979 (-6%) N/A N/A
Token:transfer_public (avm) 44,646 N/A N/A 94,400 3,916 (-3%) N/A N/A
AuthRegistry:consume (avm) 34,832 N/A N/A 94,336 3,075 (+3%) N/A N/A
FPC:pay_refund (avm) 38,561 N/A N/A 94,368 23,037 (-3%) N/A N/A
Benchmarking:create_note 1,344 9,944 480 (-1%) N/A N/A N/A N/A
SchnorrAccount:verify_private_authwit 1,280 9,944 72.7 N/A N/A N/A N/A
Token:unshield 1,376 9,944 2,721 (+1%) N/A N/A N/A N/A
FPC:fee_entrypoint_private 1,376 9,944 3,498 (+1%) N/A N/A N/A N/A

AVM Simulation

Time to simulate various public functions in the AVM.

Function time_ms bytecode_size_in_bytes
GasToken:_increase_public_balance 68.4 (-15%) 13,790
GasToken:set_portal 18.2 (-4%) 3,339 (+1%)
Token:constructor 93.9 (-5%) 23,692
FPC:constructor 69.6 (+10%) 13,592
GasToken:mint_public 50.2 (-2%) 10,158
Token:mint_public ⚠️ 585 (+709%) 19,034
Token:assert_minter_and_mint ⚠️ 65.3 (-70%) 12,925
AuthRegistry:set_authorized 33.1 7,812
FPC:prepare_fee 140 (+5%) 15,062
Token:transfer_public 77.7 (+7%) 31,218
FPC:pay_refund 153 (+3%) 25,260
Benchmarking:increment_balance 2,623 (-2%) 15,267
Token:_increase_public_balance 56.4 (+1%) 15,006
FPC:pay_refund_with_shielded_rebate 132 (-27%) 26,347

Public DB Access

Time to access various public DBs.

Function time_ms
get-nullifier-index 0.157 (-3%)

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric 1 leaves 16 leaves 64 leaves 128 leaves 256 leaves 512 leaves 1024 leaves
batch_insert_into_append_only_tree_16_depth_ms 10.4 (-2%) 16.8 (-2%) N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_count 16.8 31.7 N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_ms 0.599 (-2%) 0.514 (-2%) N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_32_depth_ms N/A N/A 48.4 (-2%) 76.4 (-2%) 132 (-4%) 246 472 (-4%)
batch_insert_into_append_only_tree_32_depth_hash_count N/A N/A 95.9 159 287 543 1,055
batch_insert_into_append_only_tree_32_depth_hash_ms N/A N/A 0.494 (-2%) 0.469 (-2%) 0.452 (-4%) 0.446 0.440 (-4%)
batch_insert_into_indexed_tree_20_depth_ms N/A N/A 59.3 (-2%) 112 (-2%) 183 (-4%) 353 (-1%) 696 (-4%)
batch_insert_into_indexed_tree_20_depth_hash_count N/A N/A 109 207 355 691 1,363
batch_insert_into_indexed_tree_20_depth_hash_ms N/A N/A 0.501 (-2%) 0.503 (-2%) 0.485 (-4%) 0.478 0.476 (-4%)
batch_insert_into_indexed_tree_40_depth_ms N/A N/A 73.0 (-1%) N/A N/A N/A N/A
batch_insert_into_indexed_tree_40_depth_hash_count N/A N/A 133 N/A N/A N/A N/A
batch_insert_into_indexed_tree_40_depth_hash_ms N/A N/A 0.519 (-1%) N/A N/A N/A N/A

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric 0 registered classes 1 registered classes
tx_size_in_bytes 85,707 670,983

Transaction size based on fee payment method

| Metric | |
| - | |

@Thunkar Thunkar marked this pull request as ready for review July 1, 2024 13:34
@@ -992,7 +992,7 @@ fn handle_storage_read(
inputs: &Vec<ValueOrArray>,
) {
// For the foreign calls we want to handle, we do not want inputs, as they are getters
assert!(inputs.len() == 1); // storage_slot
assert!(inputs.len() == 2); // output, len - but we dont use this len - its for the oracle
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a bit more detail here. In particular explain that the destination already has a length, and that is the one that will be used. (and why do we need this extra length? what does "the oracle" mean)

However... ideally we'd have an assert here that length==destination.length, if you can force the length to be known at compile time.

If you can't I guess that's life.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified a bit in the comment (first just reverted the change), but I don't see an easy way to make the second assertion, since the destination size is known at this point, but the second argument is just a memory address. Is it possible to "unwrap" it somehow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Length should be known at compile time btw, since the output of the opcode is an array explicitly (not a slice)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a question for @sirasistant

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only way for the transpiler to do it is to emit a const opcode with destination.length and an opcode that checks equality between that one and the second argument. But without scratch space it's a pain. (should I reserve scratch space at this point? it has come up several times)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is part of a larger discussion, I would leave as it is for the time being and try to get this in, since it's blocking TXE usage in public ATM (and this was already there, just removed recently)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm ok with it as it is. The question for Alvaro was more on the Noir/Brillig side. If there was any way to mark length as comptime/etc so that it gets to the transpiler as a resolved number.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'd be possible but I think there is no way to have a literal in foreign call params with the current brillig spec

@fcarreiro fcarreiro self-requested a review July 2, 2024 08:30
Copy link
Contributor

github-actions bot commented Jul 2, 2024

Changes to circuit sizes

Generated at commit: 3210f01faf28b2226c287dfc6f4cc2c2f9e0c28e, compared to commit: 6fafff6e0ccda9d1e07beb5a5e8638f75b0345c2

🧾 Summary (100% most significant diffs)

Program ACIR opcodes (+/-) % Circuit size (+/-) %
private_kernel_tail_to_public +13,124 ❌ +2.95% +347,648 ❌ +19.64%
private_kernel_tail +536 ❌ +2.50% +1,269 ❌ +0.10%
private_kernel_init -1 ✅ -0.00% 0 ➖ 0.00%

Full diff report 👇
Program ACIR opcodes (+/-) % Circuit size (+/-) %
private_kernel_tail_to_public 458,562 (+13,124) +2.95% 2,118,154 (+347,648) +19.64%
private_kernel_tail 21,972 (+536) +2.50% 1,336,812 (+1,269) +0.10%
private_kernel_init 30,686 (-1) -0.00% 390,775 (0) 0.00%

@Thunkar Thunkar merged commit de303e2 into master Jul 2, 2024
90 checks passed
@Thunkar Thunkar deleted the gj/fail_ci_on_txe_test_failure branch July 2, 2024 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants