-
Notifications
You must be signed in to change notification settings - Fork 742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add assertoor tests to CI #6192
base: unstable
Are you sure you want to change the base?
Conversation
01ced96
to
c6b10ce
Compare
c6b10ce
to
766dbd0
Compare
I don't think we need
|
…d-assertoor-github-action
…d-assertoor-github-action
the |
Can you upload the logs to the CI run artifacts? To debug failures |
logs should now be uploaded with a 3 day retention window |
path: | | ||
scripts/local_testnet/logs | ||
retention-days: 3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like these keys are invalid for this github action. (see warnings on the run)
The step below defines the artifact path and retention, I think this can be removed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We already upload the local testnet logs, did you mean to upload the assertoor logs? They could be included in the enclave logs too but i haven't checked
run_block_proposal_check: true | ||
run_blob_transaction_test: true | ||
tests: | ||
- https://raw.githubusercontent.com/ethpandaops/assertoor-test/master/assertoor-tests/big-calldata-tx-test.yaml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like the big-calldata-tx-test.yaml
is the one taking the most time (>20m) and doesn't run concurrently with the other 3 tests. Maybe we can exclude this one from the CI job and have it as a recurring test on GitHub action?
Add assertoor tests to CI
This PR adds a new step to the local_testnet CI. This new step spins up a kurtosis testnet (defined in
assertoor_network_params.yml
) and runs assertoor tests as a github action viakurtosis-assertoor-github-action
Issues w/ assertoor github action
Note that the repository ppa:rmescandon/yq is no longer maintained and does not contain the most up to date version of
yq
mikefarah/yq#1308
Installation recommendations for
yq
can be found here:https://github.com/mikefarah/yq?tab=readme-ov-file#install
I've updated the
local_testnet.yml
script to installyq
viasnap
Choosing which tests to run
The list of assertoor tests can be found here: https://github.com/ethpandaops/assertoor-test/tree/master/assertoor-tests
Theres a few tests that are broken at the momemnt (mev-block-proposal as an example) and a few that take 48 hours to complete. I've picked the most relevant tests that finish running at a reasonable time.
Theres some electra tests that we may want to add in the future