Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add rebuild pattern for bb-bin-tests to rebuild when ts folder is changed and add target folder for bb-bin-test #1640

Merged
merged 16 commits into from
Aug 18, 2023

Conversation

kevaundray
Copy link
Contributor

Please provide a paragraph or two giving a summary of the change, including relevant motivation and context.

Checklist:

Remove the checklist to signal you've completed it. Enable auto-merge if the PR is ready to merge.

  • If the pull request requires a cryptography review (e.g. cryptographic algorithm implementations) I have added the 'crypto' tag.
  • I have reviewed my diff in github, line by line and removed unexpected formatting changes, testing logs, or commented-out code.
  • Every change is related to the PR description.
  • I have linked this pull request to relevant issues (if any exist).

@kevaundray kevaundray marked this pull request as ready for review August 17, 2023 22:59
@kevaundray kevaundray changed the title chore: Add rebuild pattern for bb-bin-tests chore: Add rebuild pattern for bb-bin-tests [DRAFT] Aug 17, 2023
@kevaundray kevaundray changed the title chore: Add rebuild pattern for bb-bin-tests [DRAFT] chore: Add rebuild pattern for bb-bin-tests to rebuild when ts folder is changed[DRAFT] Aug 18, 2023
@Maddiaa0 Maddiaa0 force-pushed the kw/rebuild-pat-bb-tests branch from 14e4f2b to a26113e Compare August 18, 2023 16:16
@Maddiaa0 Maddiaa0 force-pushed the kw/rebuild-pat-bb-tests branch from 0b29286 to 807a003 Compare August 18, 2023 16:30
@kevaundray kevaundray changed the title chore: Add rebuild pattern for bb-bin-tests to rebuild when ts folder is changed[DRAFT] chore: Add rebuild pattern for bb-bin-tests to rebuild when ts folder is changed Aug 18, 2023
@kevaundray kevaundray changed the title chore: Add rebuild pattern for bb-bin-tests to rebuild when ts folder is changed chore: Add rebuild pattern for bb-bin-tests to rebuild when ts folder is changed and add target folder for bb-bin-test Aug 18, 2023
@kevaundray kevaundray requested a review from Maddiaa0 August 18, 2023 19:50
Copy link
Member

@Maddiaa0 Maddiaa0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Member

@Maddiaa0 Maddiaa0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to add an issue to re separate the workflows, other than that lgtm

@Maddiaa0
Copy link
Member

Would be good to add an issue to re separate the workflows, other than that lgtm

merging as #1665 exists

@Maddiaa0 Maddiaa0 merged commit b3ee3d9 into master Aug 18, 2023
@Maddiaa0 Maddiaa0 deleted the kw/rebuild-pat-bb-tests branch August 18, 2023 20:14
codygunton pushed a commit that referenced this pull request Jan 23, 2024
* dynamic reference to redis cluster subnet group (#1594)

Co-authored-by: PhilWindle <[email protected]>

* Created generic swapping agent for both uniswap and lido-curve bridges (#1265)

* JB/Website changes (#1554)

* Bruno changes

* Grants changes

* Update grants page, edit team page

* Fix typos, change header, footer links; noir links

* Fix sliders

* Copy changes

* Adds two RFPs

* Add Michael to Careers, typos

* Fix titles of roles

* JB/Website changes (#1612)

* Bruno changes

* Grants changes

* Update grants page, edit team page

* Fix typos, change header, footer links; noir links

* Fix sliders

* Copy changes

* Adds two RFPs

* Add Michael to Careers, typos

* Fix titles of roles

* Fix lint, replace images

* Pass the uint context to the return value of uint::at  (#1593)

* Construct return value from current context in uint::at

Fixes AztecProtocol/aztec2-internal#1433

* Add uint test which asserts context inheritance

* Move to yarn 3 plug 'n play and workspaces, pure ESM modules. (#1599)

* Transferrables and webpack to build a worker.js until we have loader chaining...

* Prod/dev conf

* Update bb imports.

* bb tests passing.

* Integrate wasm back in bb.js. Passing tests. Reduce wasm memory footprint.

* Can run halloumi and falafel.

* Falafel tests pass.

* Sdk tests pass.

* e2e.test passes.

* webpack

* bb.js yarn 3.2.2

* fix mem align

* fix bitwise warning on newer clang

* Tweaks towards wasi 16 (more to figure out)

* yarn 3.2.2

* remove encoding headers from block source client

* bb.js updates

* Blockchain test transpile to cjs for hardhat.

* Fix blockchain.

* SDK alternates proving keys.

* Falafel/halloumi start_e2e scripts args.

* Shared linting, formatting for bb.js and blockchain.

* explorer import changes. build needs fixing.

* end-to-end, falafel, halloumi linting

* Hummus just a terminal. Linting. Version upgrades.

* Linting. Bootstrapping. Wallet.

* prettier config.

* Basic working wallet.

* Fix apollo server stuff in falafel...

* account-migrator. explorer.

* Cleanup JSON stuff.

* wasabi.

* Revert jest->mocha

* Merge fixes.

* Initial zk-money.

* - Reconnect bootstrap scripts
- set prettier as js default

* - es-module style imports
- upgrade typescript
- copy wasm

* zk-money builds.

* Linting

* Dockerfile and build stuff.

* Move yarn project into own folder.

* explorer build. min cci yml.

* Introduce a build manifest to compute rebuild patterns.

* Addition of a script for auto launching tmux and running a test.

* Streamline bootstrapping.

* Add bash to falafel image.

* build image

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* Attempt to fix deploy.

* When scanning for rollups, we only cache the last block number where … (#1611)

* When scanning for rollups, we only cache the last block number where we actually found a rollup

* Fixed bug

* fix typo in zk-money app obs intial -> initial (#1633)

* Adress comments about docs (#1500)

* Adress comments about docs

* Update schnorr.md

Co-authored-by: Michael Connor <[email protected]>

* Move genesis data from bb.js to falafel (#1640)

* moved init/genesis data from bb.js to falafel along with helper functions for reading the init data.

* data files no longer need to be moved as part of bb.js webpack

* obsolete comment in bb.js env

* Moved some init helper functions back to bb.js. Renamed helper class in falafel. Fixed missing 'any' type in falafel env init.ts

* copy init account data to dest in falafel

Co-authored-by: spypsy <[email protected]>
Co-authored-by: Jonathan Bursztyn <[email protected]>
Co-authored-by: Guido Vranken <[email protected]>
Co-authored-by: Charlie Lye <[email protected]>
Co-authored-by: Charlie Lye <[email protected]>
Co-authored-by: David Banks <[email protected]>
Co-authored-by: Adrian Hamelink <[email protected]>
Co-authored-by: Michael Connor <[email protected]>
codygunton pushed a commit that referenced this pull request Jan 23, 2024
* Refactored index.ts to support multiple cli commands/modes. Split out previous logic into migrate_accounts.ts and pulled some logic out of aztec_connect.ts into helper file get_rollup_blocks.ts. Started implementation of fix_accounts.ts for accounts that can have an alias reassigned to their public key

* pulling blocks into worldstate, correct root.

* account fixer now uses SDK, generates keys and aliasHash, constructs AccountTx, performs FFT and gets signing data. Rearranged fixAccounts function

* minor tweak to fix-accounts test script

* For fixAccounts function of account-migrator, refactored the initialization of modules like worker pools, fft, prover, etc to match account_prover.test.ts. This fixed a memory access out-of-bounds error in WASM. Am now computing schnorr signature properly, generating account proof, and verifying account proof

* Updates to fix-accounts-local.sh so it detects when falafel is ready, parses rollup address, runs e2e test and then runs fixAccounts test locally. Can launch full test including ganache, 2 halloumis and a falafel all with a single run of this script

* Began making shift over to Halloumi proofs while investigating why falafel is rejecting proofs generated directly by the account-migrator

* Updated barretenberg rollup_cli and account proofs to accept account txs from halloumi for proof generation. Updated falafel to accept second-class transactions that do not pay a fee (still need to remove them from get_pending_txs and enforce that they only fill up empty space in blocks. fixAccounts is now submitting account txs to halloumi for proof generation before submitting them to falafel for validation and inclusion in rollup blocks. Falafel now has an e2e 'dev' start mode that sets up for e2e but also runs in dev mode for continuous rebuilding

* some cleanup in the account fixer. Started adding code (commented out) to write/read account proofs to/from json files

* Added logic for writing/reading account proofs to json files in batches before sending to falafel. Moved createFundedWalletProvider to utility file as is done for e2e tests. General cleanup in fix_accounts.

* Moved sendSecondClassTxs into a 'limited' rollup provider instantiated in account fixer.

* submitting actual batches of 2nd class txs to falafel

* Moved sendSecondClassTxs into a new class. Moved all account fixing logic into AccountFixer class in account_fixer.ts

* log fix

* Added invertPublicKey wasm bindings. Creating new account first to then be used by many migration TXs for account fixing/hardening

* Fixed account-migrator package deps. Began changes to pull real accounts from mainnet for account hardening/fixing

* broke yarn lock file after merge

* 2nd class txs now only fill up empty slots in rollup block. account fixer force flushes the rollup when in test mode

* rename account fixer to hardener

* added validateAuth for second class txs

* fix account.cpp info text

* rename function to get rollup publish conditions in rollup coordinator. Updated core SDK's pendingAliases to use a Map so that when multiple migration txs are submitted with the same alias, it only updates the SDK with the last tx's alias/acc-public-key pair

* More private helper functions to break up long code blocks in hardener. Added some function header comments.

* fixed falafel bug and broken falafel tests

* separate modes for generation of proofs and submission of proofs

* hardener now hardens 'initial' accounts. Added 'verifyHardened' command

* minor cleanup in sdk and halloumi

* hardener readme and script update

* added pendingSecondClassTxs to status endpoint

* Added falafel-monitoring mode for hardener. Hardener now ensures a unique set of accounts before generating proofs. Other fixes and cleanup to hardener

* hardener cleanup

* debugger working properly

* Added schnorr public key inversion test to barretenberg JS. Added inversion function comments to barretenberg and bJS

* prettier for barretenberg js

* Typos in zk-money and bberg JS (intial -> initial). Split hardener commands into subcommands `harden createHardener` etc. Updated migrator README

* comments and prints cleanup

* revert falafel package.json and start_e2e.sh back to defi-bridge-project version. Commented account hardener and cleaned up harden-command modes (new full-sequence mode that performs all harden steps).

* Tweaks to falafel after code review, bug in rollup_db getTotalTxCount, revert minor/style circuit change. Large updates to Account Hardener after review. Some additional comments and TODOs

* include second class tx count (total and per type) in rollup profile print.

* comment local hardener test script

* fixed bug where second class txs could surpass block-slot limit

* added rollup coordinator tests to check that: 1. rollup does not publish when filled with second class txs, 2. second-class txs are omitted when block is full with 1st class, 3. second-class txs are included when there is empty space and a rollup is going to publish anyway, 3. second class txs are omitted if they would surpass gas/call-data limits

* prettier rollup_coordinator

* compiler warning in rollup coord test

* Hardener now always pulls ALL accounts from ALL blocks and segregates the target accounts for proof generation/verification

* hardener constants updated (batch size and max second-class txs in falafel. Format fix in rolllup profile printer. Added metrics gauge for 2nd-class txs

* Was able to reduce second-class-tx-related changeset in rollup_coordinator.test.ts via use of mockResolvedValueOnce to inject second class txs into rollupDb

* improved hardener script. cleanup in hardener logging

* various yarn package related issues resolved after large merge

* bugs found by ci in barretenberg and falafel after large merge

* type fix in rollup coordinator test

* cleanup for db/account-hardener PR

* revert zkmoney change not appropriate in hardener PR

* minor style fix in account hardener local testing script

* changed terminology: invert -> negate (for negation of public key Y coord)

* fix account hardener after PR #1640

* simplify public key negation

* fixes after merge update

* fix account hardener after Crs->NetCrs rename
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants