-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: tone down ci some more #492
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like the time for the invariant tests has decreased significantly, but not so much for the fork and coverage. I believe this issue is caused by the foundry.
https://github.com/sablierhq/v2-core/actions/runs/4984576283
That's odd .. the need to communicate to the RPC might explain the fork tests, but the coverage job should have been faster because of the $ forge coverage --no-match-path "test/fork/**/*.sol" --report lcov I'll investigate some more before merging this PR. |
989075a
to
b055ddc
Compare
Following up with my investigation on why the coverage job is taking so long (>1 hour) still.
I proceeded to run the coverage command on my local machine, and noticed the same slow behavior. This has led me to conclude that there is a bug in Foundry, which I reported here: In our PR, I updated the coverage job to include only unit and fuzz tests, like this: $ forge coverage --match-path "test/{fuzz,unit}/**/*.sol" --report lcov @andreivladbrg - let me know if this is good to merge now. |
I've also investigated why the fork tests take so long (>1 hour). Here are my findings:
We might be able to speed up the CI run time by upgrading to larger GitHub runners with more cores, although this would require us to upgrade to a paid GitHub plan (e.g. Team). Similarly, we might get rate limited less often if we upgraded to a higher plan on Alchemy and/ or ChainNodes. Anyway. In the meantime, I will simply lower the number of fuzz runs in the Update: it is also unclear how Foundry handles RPC retries. See foundry-rs/foundry#4958. |
ci: disable fork tests in coverage job test: remove "fuzz" and "invariant" config for "test-optimized" profile
Seems like this approach is working: the fork tests take ~15 minutes to run now: https://github.com/sablierhq/v2-core/actions/runs/5002102460/jobs/8962236042?pr=492 Merging. |
We didn't tone CI down enough in #474. Some workflows are still taking a long time to run:
https://github.com/sablierhq/v2-core/actions/runs/4984025835/jobs/8921863702?pr=490