-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: snapshot fuzz tests using deterministic seed #2591
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice
Hmm, have to think about this a bit more. It slowed down coverage quite a lot (26s on solmate). I guess that's how slow coverage is going to be with 100% fuzz test support... surely there's a way to optimize |
Feels unrelated though right? i.e. in order to get good coverage metrics we'd still want our fuzz tests included (using the det. seed) Have you tried profiling coverage? Suspect a lot would be in the IC/PC/stack/memory instrumentation? |
Yeah I'm sure there are a lot of performance gains to be had in optimizing our IC/PC thing, both for the debugger and for coverage. FWIW it was really fast before fuzz tests were enabled. I have some ideas for how to optimize it I'll look at tomorrow, since I am going to be looking at full fuzz support for coverage anyway I haven't tried profiling it yet, though |
do you feel OK with merging this, so we get snapshots "finished", and we can investigate coverage separately given we're not 100% there yet anyway? |
Yep, sg |
Motivation
We introduced
--include-fuzz-tests
to make gas snapshots less prone to change from fuzz RNG because we had no way to make the fuzzer deterministic. Now that the fuzzer is deterministic we can remove the flag and include fuzz tests alwaysSolution
Removes
--include-fuzz-tests
and uses a deterministic seed forforge snapshot
runs. This also cleans up the code a bit, since we introduced a variable we had to carry into many files to disable fuzz tests.Closes #2553