feat: rust toolchain binary cataloger #1601

westonsteimel · 2023-02-21T19:17:49Z

Adds binary classifier for the rust toolchain for linux and mac

github-actions · 2023-02-21T19:23:52Z

Benchmark Test Results

Benchmark results from the latest changes vs base branch

goos: linux
goarch: amd64
pkg: github.com/anchore/syft/test/integration
cpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
                                                          │ ./.tmp/benchmark-3d006a4.txt │
                                                          │            sec/op            │
ImagePackageCatalogers/alpmdb-cataloger-2                                   11.79m ± 23%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                             893.4µ ±  3%
ImagePackageCatalogers/python-package-cataloger-2                           3.032m ±  1%
ImagePackageCatalogers/php-composer-installed-cataloger-2                   691.9µ ±  1%
ImagePackageCatalogers/javascript-package-cataloger-2                       372.8µ ±  1%
ImagePackageCatalogers/dpkgdb-cataloger-2                                   513.5µ ±  1%
ImagePackageCatalogers/rpm-db-cataloger-2                                   484.8µ ±  1%
ImagePackageCatalogers/java-cataloger-2                                     10.76m ±  1%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.223µ ±  3%
ImagePackageCatalogers/apkdb-cataloger-2                                    509.4µ ±  2%
ImagePackageCatalogers/go-module-binary-cataloger-2                         18.94µ ±  1%
ImagePackageCatalogers/dotnet-deps-cataloger-2                              971.4µ ±  3%
ImagePackageCatalogers/portage-cataloger-2                                  322.7µ ±  2%
ImagePackageCatalogers/sbom-cataloger-2                                     105.3µ ±  1%
ImagePackageCatalogers/binary-cataloger-2                                   179.2µ ±  1%
geomean                                                                     465.2µ

                                                          │ ./.tmp/benchmark-3d006a4.txt │
                                                          │             B/op             │
ImagePackageCatalogers/alpmdb-cataloger-2                                   5.062Mi ± 0%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                             141.9Ki ± 0%
ImagePackageCatalogers/python-package-cataloger-2                           947.3Ki ± 0%
ImagePackageCatalogers/php-composer-installed-cataloger-2                   155.9Ki ± 0%
ImagePackageCatalogers/javascript-package-cataloger-2                       95.72Ki ± 0%
ImagePackageCatalogers/dpkgdb-cataloger-2                                   144.7Ki ± 0%
ImagePackageCatalogers/rpm-db-cataloger-2                                   170.5Ki ± 0%
ImagePackageCatalogers/java-cataloger-2                                     2.720Mi ± 0%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                     1.523Ki ± 0%
ImagePackageCatalogers/apkdb-cataloger-2                                    123.0Ki ± 0%
ImagePackageCatalogers/go-module-binary-cataloger-2                         3.102Ki ± 0%
ImagePackageCatalogers/dotnet-deps-cataloger-2                              314.7Ki ± 0%
ImagePackageCatalogers/portage-cataloger-2                                  75.47Ki ± 0%
ImagePackageCatalogers/sbom-cataloger-2                                     13.05Ki ± 0%
ImagePackageCatalogers/binary-cataloger-2                                   26.94Ki ± 0%
geomean                                                                     108.6Ki

                                                          │ ./.tmp/benchmark-3d006a4.txt │
                                                          │          allocs/op           │
ImagePackageCatalogers/alpmdb-cataloger-2                                    86.71k ± 0%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.159k ± 0%
ImagePackageCatalogers/python-package-cataloger-2                            15.48k ± 0%
ImagePackageCatalogers/php-composer-installed-cataloger-2                    3.458k ± 0%
ImagePackageCatalogers/javascript-package-cataloger-2                        1.253k ± 0%
ImagePackageCatalogers/dpkgdb-cataloger-2                                    2.646k ± 0%
ImagePackageCatalogers/rpm-db-cataloger-2                                    3.759k ± 0%
ImagePackageCatalogers/java-cataloger-2                                      38.26k ± 0%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                       40.00 ± 0%
ImagePackageCatalogers/apkdb-cataloger-2                                     3.252k ± 0%
ImagePackageCatalogers/go-module-binary-cataloger-2                           101.0 ± 0%
ImagePackageCatalogers/dotnet-deps-cataloger-2                               5.011k ± 0%
ImagePackageCatalogers/portage-cataloger-2                                   1.487k ± 0%
ImagePackageCatalogers/sbom-cataloger-2                                       392.0 ± 0%
ImagePackageCatalogers/binary-cataloger-2                                     772.0 ± 0%
geomean                                                                      2.205k

kzantow

It might be nice to cut down the size of some of the test fixtures (some are in the MB range)

syft/pkg/cataloger/binary/default_classifiers.go

wagoodman

...some are in the MB range....

it seems like there may be an extra addition to the gitignore needed for the busybox image fixture tar

Alex requested changes

westonsteimel · 2023-02-22T19:45:41Z

...some are in the MB range....

it seems like there may be an extra addition to the gitignore needed for the busybox image fixture tar

I think it might already be excluded - I had to force add my test cases since they'd normally be excluded and must have accidentally picked up that as well

kzantow

The /rust-1.67.1/toolchains/stable-x86_64-unknown-linux-musl/lib/libstd-86aefecbddda356d.so is still 1.25 MB, it would be nice to just copy the small (maybe < 100 bytes) binary bits that are required for matching. Other than that, 👍

Signed-off-by: Weston Steimel <[email protected]>

spiffcs · 2023-02-27T19:52:42Z

@wagoodman I think all new fixture changes are now only in the <30kb size now:

Did you have other changes requested for this PR? Otherwise LGTM

wagoodman · 2023-03-01T14:52:04Z

syft/pkg/cataloger/binary/default_classifiers.go

@@ -196,4 +196,24 @@ var defaultClassifiers = []classifier{
 		Package: "postgresql",
 		PURL:    mustPURL("pkg:generic/postgresql@version"),
 	},
+	{
+		Class:    "rust-standard-library-linux",
+		FileGlob: "**/libstd-????????????????.so",


good call, that is more specific and not too crazy for the index parsing in stereoscope 🙌

Signed-off-by: Weston Steimel <[email protected]>

westonsteimel force-pushed the rust-toolchain-cataloger branch from c2e4e78 to a440961 Compare February 21, 2023 19:32

westonsteimel added the enhancement New feature or request label Feb 21, 2023

westonsteimel requested a review from a team February 21, 2023 20:47

kzantow previously approved these changes Feb 22, 2023

View reviewed changes

syft/pkg/cataloger/binary/default_classifiers.go Outdated Show resolved Hide resolved

wagoodman requested changes Feb 22, 2023

View reviewed changes

westonsteimel force-pushed the rust-toolchain-cataloger branch from a440961 to 93ae287 Compare February 22, 2023 19:43

westonsteimel force-pushed the rust-toolchain-cataloger branch 2 times, most recently from 731abd9 to 3090714 Compare February 22, 2023 20:02

westonsteimel requested review from wagoodman and kzantow February 24, 2023 18:24

kzantow approved these changes Feb 24, 2023

View reviewed changes

feat: rust toolchain binary cataloger

5e70c7d

Signed-off-by: Weston Steimel <[email protected]>

westonsteimel force-pushed the rust-toolchain-cataloger branch from 3090714 to 5e70c7d Compare February 24, 2023 20:49

wagoodman reviewed Mar 1, 2023

View reviewed changes

wagoodman approved these changes Mar 1, 2023

View reviewed changes

westonsteimel merged commit e92b0fa into main Mar 1, 2023

westonsteimel deleted the rust-toolchain-cataloger branch March 1, 2023 14:53

This was referenced Mar 14, 2023

build(deps): bump github.com/anchore/syft from 0.72.0 to 0.75.0 docker/buildkit-syft-scanner#41

Closed

Bump github.com/anchore/syft from 0.71.0 to 0.75.0 valllabh/security-cli#18

Closed

GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024

feat: rust toolchain binary cataloger (anchore#1601)

e3ef0c3

Signed-off-by: Weston Steimel <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: rust toolchain binary cataloger #1601

feat: rust toolchain binary cataloger #1601

westonsteimel commented Feb 21, 2023

github-actions bot commented Feb 21, 2023 •

edited

Loading

kzantow left a comment

wagoodman left a comment

westonsteimel commented Feb 22, 2023

kzantow left a comment

spiffcs commented Feb 27, 2023

wagoodman Mar 1, 2023

feat: rust toolchain binary cataloger #1601

feat: rust toolchain binary cataloger #1601

Conversation

westonsteimel commented Feb 21, 2023

github-actions bot commented Feb 21, 2023 • edited Loading

Benchmark Test Results

kzantow left a comment

Choose a reason for hiding this comment

wagoodman left a comment

Choose a reason for hiding this comment

westonsteimel commented Feb 22, 2023

kzantow left a comment

Choose a reason for hiding this comment

spiffcs commented Feb 27, 2023

wagoodman Mar 1, 2023

Choose a reason for hiding this comment

github-actions bot commented Feb 21, 2023 •

edited

Loading