[gitpython] Fix Broken Build #11763
Closed
[gitpython] Fix Broken Build #11763
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The pre-installed version of `pip` (19.x) was outdated and unable to parse the `pyproject.toml` syntax during the install step in `build.sh` causing the script to error out and crash. Upgrading `pip` to the latest version in the project image resolves the issue and allows the installation to complete. [1]: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67399
This project has a deep dependency graph which caused the call to atheris.instrament_all() slow down the fuzzers significantly as it instramented 4,000+ functions, many of which were not relevant to the specific fuzz target APIs. Replacing it with a with-statement that only instraments the target code provides significant efficency benefits in startup time and execution speed. Dictionaries and seed_corpus zips provide additional efficiency boosts. This was made evident by the new exceptions raised by the fuzz input data which are now handled accordingly.
|
DaveLak is a new contributor to projects/gitpython. The PR must be approved by known contributors before it can be merged. |
Byron
approved these changes
Mar 31, 2024
|
Closing this in favor of a forthcoming PR to migrate the fuzz harnesses upstream. |
DaveLak
added a commit
to DaveLak/GitPython
that referenced
this pull request
Apr 12, 2024
Migrates the OSS-Fuzz tests and setup scripts from the OSS-Fuzz repository to GitPython's repo as discussed here: gitpython-developers#1887 (comment) These files include the changes that were originally proposed in: google/oss-fuzz#11763 Additional changes include: - A first pass at documenting the contents of the fuzzing set up in a dedicated README.md - Adding the dictionary files to this repo for improved visibility. Seed corpra zips are still located in an external repo pending further discussion regarding where those should live in the long term.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes ClusterFuzz issues 67399 and 55299
Issue 67399: gitpython: Fuzzing build failure
Since: 2024-03-11
The Problem
The pre-installed version of
pip(19.2.3) was outdated and unable to parse thepyproject.tomlsyntax during the install step inbuild.shcausing the script to error out and crash.The Solution
Upgrading
pipto the latest version in the project image resolves the issue and allows the installation to complete.Issue 55299: gitpython: Coverage build failure
Since: 2023-01-21
The Problem
(my hypothesis at least)
I believe the root of the issue here was caused by fuzzer initialization and execution taking too long for the actual run to generate a meaningful corpus. I suspect this because:
atheris.instrament_all()to instrument 4,000+ functions before the fuzzer execution could begin which was causing a significant delay (on my local machine, at least) before actual test execution would start.The Solution(-ish)
The commit message on 908ba9c should sum it up, but the TL;DR is I reduced the scope of instrumented functions to align closer with the APIs being fuzzed and added dictionaries and seed corpra which provided promising results locally.
fuzz_tree.pyis still slow as far as average_exec_per_sec, but startup is quicker and with the seed data it gets close to its coverage depth fairly quickly as well.