Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixup libafl_fuzzbench-based fuzzers #2032

Merged
merged 6 commits into from
Sep 19, 2024

Conversation

addisoncrump
Copy link
Contributor

Fixes #2019.

@addisoncrump addisoncrump marked this pull request as ready for review August 19, 2024 02:06
@addisoncrump
Copy link
Contributor Author

addisoncrump commented Aug 19, 2024

This PR also restores the mruby bug benchmark. Potentially I could bring back other benchmarks for these fuzzers.

@DonggeLiu, would you think it appropriate to run a fuzzbench run for these fuzzers + a working symbolic fuzzer + baselines (AFL++ and libafl at least) against mruby? After #2028 lands.

@DonggeLiu
Copy link
Contributor

Yep sure, I would love to see the results too. Thanks for doing this, @addisoncrump!

If you are interested in running them on more bug-based benchmarks, they (and their results) can be found on sbft23 branch and #2022. @Dammo3 kindly added some benchmarks recently.

Two minor things:

  1. We conventionally append the commit ID after the directory name of bug-based benchmarks, e.g., https://github.com/google/fuzzbench/tree/master/benchmarks/bloaty_fuzz_target_52948c

  2. How about changin the type to type: bug:

@@ -33,7 +33,7 @@ jobs:
- libxslt_xpath
- mbedtls_fuzz_dtlsclient
- mbedtls_fuzz_dtlsclient_7c6b0e
- mruby_mruby_fuzzer
- mruby_mruby_fuzzer_8c8bbd
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DonggeLiu Maybe something worth noting: the CI will not fail if the benchmark doesn't exist!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, I did not know this, thanks!
I reckon this is low priority because it does not affect checking fuzzers or running experiments.
WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reckon it's low priority as well, but maybe something to keep in mind since it might inadvertently mark a fuzzer as "working" for a non-existent benchmark.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, created this: #2037

@renatahodovan
Copy link

@addisoncrump Thanks for working on this! Is it planned to merge it anytime soon?

@addisoncrump
Copy link
Contributor Author

This is ready to be merged.

@DonggeLiu DonggeLiu merged commit 915cd23 into google:master Sep 19, 2024
32 checks passed
ardier pushed a commit to ardier/fuzzbench that referenced this pull request Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Grammar fuzzer build failed
3 participants