Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relax error conditions for catalogers #1492

Merged
merged 3 commits into from
Jan 20, 2023
Merged

Conversation

wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Jan 19, 2023

This PR adjusts the binary cataloger to compartmentalize errors to individual classifiers and continue to the next classifier when an error occurs.

This prevents errors from being fatal, such as:

[0001] TRACE cataloging complete cataloger=portage-cataloger
[0001] TRACE cataloging started cataloger=sbom-cataloger
[0001] DEBUG discovered 0 packages cataloger=sbom-cataloger
[0001] TRACE cataloging complete cataloger=sbom-cataloger
[0001] TRACE cataloging started cataloger=binary-cataloger
[0001]  WARN error while cataloging cataloger=binary-cataloger
No packages discovered
2023/01/19 13:12:27 error during command execution: 1 error occurred:
	* failed to resolve files by glob (**/python*): cycle during symlink resolution

@wagoodman wagoodman added the bug Something isn't working label Jan 19, 2023
@github-actions
Copy link

github-actions bot commented Jan 19, 2023

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux
goarch: amd64
pkg: github.com/anchore/syft/test/integration
cpu: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
                                                          │ ./.tmp/benchmark-3f965a0.txt │
                                                          │            sec/op            │
ImagePackageCatalogers/alpmdb-cataloger-2                                   14.43m ±  1%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                             1.595m ± 14%
ImagePackageCatalogers/python-package-cataloger-2                           4.053m ±  1%
ImagePackageCatalogers/php-composer-installed-cataloger-2                   1.325m ±  1%
ImagePackageCatalogers/javascript-package-cataloger-2                       938.8µ ±  1%
ImagePackageCatalogers/dpkgdb-cataloger-2                                   1.068m ±  1%
ImagePackageCatalogers/rpm-db-cataloger-2                                   1.578m ±  1%
ImagePackageCatalogers/java-cataloger-2                                     17.16m ±  1%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.196µ ±  1%
ImagePackageCatalogers/apkdb-cataloger-2                                    1.078m ±  0%
ImagePackageCatalogers/go-module-binary-cataloger-2                         17.21µ ±  1%
ImagePackageCatalogers/dotnet-deps-cataloger-2                              1.644m ±  3%
ImagePackageCatalogers/portage-cataloger-2                                  913.3µ ±  5%
ImagePackageCatalogers/sbom-cataloger-2                                     5.594m ±  1%
ImagePackageCatalogers/binary-cataloger-2                                   7.889m ±  1%
geomean                                                                     1.265m

                                                          │ ./.tmp/benchmark-3f965a0.txt │
                                                          │             B/op             │
ImagePackageCatalogers/alpmdb-cataloger-2                                   5.041Mi ± 0%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                             204.5Ki ± 0%
ImagePackageCatalogers/python-package-cataloger-2                           953.6Ki ± 0%
ImagePackageCatalogers/php-composer-installed-cataloger-2                   217.2Ki ± 0%
ImagePackageCatalogers/javascript-package-cataloger-2                       158.7Ki ± 0%
ImagePackageCatalogers/dpkgdb-cataloger-2                                   199.9Ki ± 0%
ImagePackageCatalogers/rpm-db-cataloger-2                                   300.4Ki ± 0%
ImagePackageCatalogers/java-cataloger-2                                     3.362Mi ± 0%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                     1.375Ki ± 0%
ImagePackageCatalogers/apkdb-cataloger-2                                    181.4Ki ± 0%
ImagePackageCatalogers/go-module-binary-cataloger-2                         1.809Ki ± 0%
ImagePackageCatalogers/dotnet-deps-cataloger-2                              375.8Ki ± 0%
ImagePackageCatalogers/portage-cataloger-2                                  138.1Ki ± 0%
ImagePackageCatalogers/sbom-cataloger-2                                     724.7Ki ± 0%
ImagePackageCatalogers/binary-cataloger-2                                   1.028Mi ± 0%
geomean                                                                     219.0Ki

                                                          │ ./.tmp/benchmark-3f965a0.txt │
                                                          │          allocs/op           │
ImagePackageCatalogers/alpmdb-cataloger-2                                    85.84k ± 0%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                              4.350k ± 0%
ImagePackageCatalogers/python-package-cataloger-2                            16.82k ± 0%
ImagePackageCatalogers/php-composer-installed-cataloger-2                    5.607k ± 0%
ImagePackageCatalogers/javascript-package-cataloger-2                        3.431k ± 0%
ImagePackageCatalogers/dpkgdb-cataloger-2                                    4.578k ± 0%
ImagePackageCatalogers/rpm-db-cataloger-2                                    8.314k ± 0%
ImagePackageCatalogers/java-cataloger-2                                      58.33k ± 0%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                       32.00 ± 0%
ImagePackageCatalogers/apkdb-cataloger-2                                     5.332k ± 0%
ImagePackageCatalogers/go-module-binary-cataloger-2                           55.00 ± 0%
ImagePackageCatalogers/dotnet-deps-cataloger-2                               7.227k ± 0%
ImagePackageCatalogers/portage-cataloger-2                                   3.682k ± 0%
ImagePackageCatalogers/sbom-cataloger-2                                      25.31k ± 0%
ImagePackageCatalogers/binary-cataloger-2                                    36.81k ± 0%
geomean                                                                      5.244k

Signed-off-by: Christopher Phillips <[email protected]>
@wagoodman wagoodman enabled auto-merge (squash) January 19, 2023 23:25
Signed-off-by: Christopher Phillips <[email protected]>
@wagoodman wagoodman merged commit 0f75f97 into main Jan 20, 2023
@wagoodman wagoodman deleted the binary-cataloger-suppress-errors branch January 20, 2023 00:28
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
* binary cataloger should continue on errors

Signed-off-by: Alex Goodman <[email protected]>

* test: add redirect for cmd stderr stdout

Signed-off-by: Christopher Phillips <[email protected]>

* test: image update for test failure

Signed-off-by: Christopher Phillips <[email protected]>

Signed-off-by: Alex Goodman <[email protected]>
Signed-off-by: Christopher Phillips <[email protected]>
Co-authored-by: Christopher Phillips <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants