-
Notifications
You must be signed in to change notification settings - Fork 594
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* initial v4 schema setup Signed-off-by: Weston Steimel <[email protected]> * update v3 => v4 for unit tests -- did NOT update - grype/db/v3/* Signed-off-by: Christopher Phillips <[email protected]> * use nullable string in sqlite so null values get represented correctly Signed-off-by: Weston Steimel <[email protected]> * add missing unit test case for dotnet Signed-off-by: Weston Steimel <[email protected]> * Add db writer function for calling sqlite vacuum Signed-off-by: Weston Steimel <[email protected]> * adding normalization of package names at database adapter layer Signed-off-by: Weston Steimel <[email protected]> * refactor namespaces for v4 Signed-off-by: Weston Steimel <[email protected]> * update v4 stuff to use sqlite fork Signed-off-by: Weston Steimel <[email protected]> * Namespace should satisfy Stringer interface Signed-off-by: Weston Steimel <[email protected]> * normalize CPEs before comparison Signed-off-by: Weston Steimel <[email protected]> * vulnerability exclusion => vulnerability match exclusion Signed-off-by: Weston Steimel <[email protected]> * updates to vulnerability match exclusion models Signed-off-by: Weston Steimel <[email protected]> * add initial vulnerability match exclusion store unit tests Signed-off-by: Weston Steimel <[email protected]> * make vuln match exclusion constraints nullable Signed-off-by: Weston Steimel <[email protected]> * move vuln match namespace into constraints object and refactor Signed-off-by: Weston Steimel <[email protected]> * check db match constraints to ensure there aren't any unknown fields and add json hints Signed-off-by: Weston Steimel <[email protected]> * ensure we only keep compatible match exclusion constraints Signed-off-by: Weston Steimel <[email protected]> * use omitempty on all match exclusion structs Signed-off-by: Weston Steimel <[email protected]> * remove db v4 schema resolver and namespace types Signed-off-by: Alex Goodman <[email protected]> * rename Vacuum to Close Signed-off-by: Alex Goodman <[email protected]> * lint fixes + remove panic on vuln provider creation Signed-off-by: Alex Goodman <[email protected]> * WIP match exclusions Signed-off-by: Weston Steimel <[email protected]> * build list of ignore rules from v4 db records Signed-off-by: Weston Steimel <[email protected]> * quick attempt at a new uber object Signed-off-by: Weston Steimel <[email protected]> * just pass around the full object for now to quickly get to a usable state Signed-off-by: Weston Steimel <[email protected]> * fix panic when no vuln db loaded Signed-off-by: Weston Steimel <[email protected]> * use interfaces for db.store function signatures Signed-off-by: Alex Goodman <[email protected]> * Flatten the match exclusion constraint model to simplify logic Signed-off-by: Weston Steimel <[email protected]> * updating some tests Signed-off-by: Weston Steimel <[email protected]> * fix panic when no db update possible Signed-off-by: Weston Steimel <[email protected]> * more tests Signed-off-by: Weston Steimel <[email protected]> * WIP fixing match exclusion constraint usability and json mapping logic Signed-off-by: Weston Steimel <[email protected]> * add v4 db diff logic (excluding vulnerability_match_exclusion data for now) Signed-off-by: Weston Steimel <[email protected]> * lint fix Signed-off-by: Weston Steimel <[email protected]> * update integration tests Signed-off-by: Weston Steimel <[email protected]> * nvd -> nvd:cpe namespace updates Signed-off-by: Weston Steimel <[email protected]> * ensure test store uses v4 normalized names Signed-off-by: Weston Steimel <[email protected]> * set the grype db update url to staging for v4 Signed-off-by: Weston Steimel <[email protected]> * prevent more segfaults on database open Signed-off-by: Weston Steimel <[email protected]> * add continue when unable to load ignore rules Signed-off-by: Weston Steimel <[email protected]> * remove db.Status from the Store object Signed-off-by: Weston Steimel <[email protected]> * fix compare_sbom_input_vs_lib_test.go Signed-off-by: Weston Steimel <[email protected]> * remove staging endpoint now that v4 is published Signed-off-by: Weston Steimel <[email protected]> Co-authored-by: Christopher Phillips <[email protected]> Co-authored-by: Alex Goodman <[email protected]>
- Loading branch information
3 people
authored
Jul 5, 2022
1 parent
75a7e54
commit 44032c5
Showing
83 changed files
with
5,012 additions
and
218 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| package sqlite | ||
|
|
||
| import ( | ||
| "database/sql" | ||
| "encoding/json" | ||
| ) | ||
|
|
||
| type NullString struct { | ||
| sql.NullString | ||
| } | ||
|
|
||
| func NewNullString(s string, valid bool) NullString { | ||
| return NullString{ | ||
| sql.NullString{ | ||
| String: s, | ||
| Valid: valid, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| func ToNullString(v any) NullString { | ||
| nullString := NullString{} | ||
| nullString.Valid = false | ||
|
|
||
| if v != nil { | ||
| var stringValue string | ||
|
|
||
| if s, ok := v.(string); ok { | ||
| stringValue = s | ||
| } else { | ||
| vBytes, err := json.Marshal(v) | ||
| if err != nil { | ||
| // TODO: just no | ||
| panic(err) | ||
| } | ||
|
|
||
| stringValue = string(vBytes) | ||
| } | ||
|
|
||
| if stringValue != "null" { | ||
| nullString.String = stringValue | ||
| nullString.Valid = true | ||
| } | ||
| } | ||
|
|
||
| return nullString | ||
| } | ||
|
|
||
| func (v NullString) ToByteSlice() []byte { | ||
| if v.Valid { | ||
| return []byte(v.String) | ||
| } | ||
|
|
||
| return []byte("null") | ||
| } | ||
|
|
||
| func (v NullString) MarshalJSON() ([]byte, error) { | ||
| if v.Valid { | ||
| return json.Marshal(v.String) | ||
| } | ||
|
|
||
| return json.Marshal(nil) | ||
| } | ||
|
|
||
| func (v *NullString) UnmarshalJSON(data []byte) error { | ||
| if data != nil && string(data) != "null" { | ||
| v.Valid = true | ||
| v.String = string(data) | ||
| } else { | ||
| v.Valid = false | ||
| } | ||
| return nil | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| package sqlite | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/assert" | ||
| ) | ||
|
|
||
| func TestToNullString(t *testing.T) { | ||
| tests := []struct { | ||
| name string | ||
| input any | ||
| expected NullString | ||
| }{ | ||
| { | ||
| name: "Nil input", | ||
| input: nil, | ||
| expected: NullString{}, | ||
| }, | ||
| { | ||
| name: "String null", | ||
| input: "null", | ||
| expected: NullString{}, | ||
| }, | ||
| { | ||
| name: "Other string", | ||
| input: "Hello there {}", | ||
| expected: NewNullString("Hello there {}", true), | ||
| }, | ||
| { | ||
| name: "Single struct with all fields populated", | ||
| input: struct { | ||
| Boolean bool `json:"boolean"` | ||
| String string `json:"string"` | ||
| Integer int `json:"integer"` | ||
| InnerStruct struct { | ||
| StringList []string `json:"string_list"` | ||
| } `json:"inner_struct"` | ||
| }{ | ||
| Boolean: true, | ||
| String: "{}", | ||
| Integer: 1034, | ||
| InnerStruct: struct { | ||
| StringList []string `json:"string_list"` | ||
| }{ | ||
| StringList: []string{"a", "b", "c"}, | ||
| }, | ||
| }, | ||
| expected: NewNullString(`{"boolean":true,"string":"{}","integer":1034,"inner_struct":{"string_list":["a","b","c"]}}`, true), | ||
| }, | ||
| { | ||
| name: "Single struct with one field populated", | ||
| input: struct { | ||
| Boolean bool `json:"boolean"` | ||
| String string `json:"string"` | ||
| Integer int `json:"integer"` | ||
| InnerStruct struct { | ||
| StringList []string `json:"string_list"` | ||
| } `json:"inner_struct"` | ||
| }{ | ||
| Boolean: true, | ||
| }, | ||
| expected: NewNullString(`{"boolean":true,"string":"","integer":0,"inner_struct":{"string_list":null}}`, true), | ||
| }, | ||
| { | ||
| name: "Single struct with one field populated omit empty", | ||
| input: struct { | ||
| Boolean bool `json:"boolean,omitempty"` | ||
| String string `json:"string,omitempty"` | ||
| Integer int `json:"integer,omitempty"` | ||
| InnerStruct struct { | ||
| StringList []string `json:"string_list,omitempty"` | ||
| } `json:"inner_struct,omitempty"` | ||
| }{ | ||
| Boolean: true, | ||
| }, | ||
| expected: NewNullString(`{"boolean":true,"inner_struct":{}}`, true), | ||
| }, | ||
| { | ||
| name: "Array of structs", | ||
| input: []struct { | ||
| Boolean bool `json:"boolean,omitempty"` | ||
| String string `json:"string,omitempty"` | ||
| Integer int `json:"integer,omitempty"` | ||
| }{ | ||
| { | ||
| Boolean: true, | ||
| String: "{}", | ||
| Integer: 1034, | ||
| }, | ||
| { | ||
| String: "[{}]", | ||
| }, | ||
| { | ||
| Integer: -5000, | ||
| Boolean: false, | ||
| }, | ||
| }, | ||
| expected: NewNullString(`[{"boolean":true,"string":"{}","integer":1034},{"string":"[{}]"},{"integer":-5000}]`, true), | ||
| }, | ||
| } | ||
|
|
||
| for _, test := range tests { | ||
| t.Run(test.name, func(t *testing.T) { | ||
| result := ToNullString(test.input) | ||
| assert.Equal(t, test.expected, result) | ||
| }) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| package db | ||
|
|
||
| import ( | ||
| "fmt" | ||
|
|
||
| grypeDB "github.com/anchore/grype/grype/db/v4" | ||
| "github.com/anchore/grype/grype/match" | ||
| ) | ||
|
|
||
| var _ match.ExclusionProvider = (*MatchExclusionProvider)(nil) | ||
|
|
||
| type MatchExclusionProvider struct { | ||
| reader grypeDB.VulnerabilityMatchExclusionStoreReader | ||
| } | ||
|
|
||
| func NewMatchExclusionProvider(reader grypeDB.VulnerabilityMatchExclusionStoreReader) *MatchExclusionProvider { | ||
| return &MatchExclusionProvider{ | ||
| reader: reader, | ||
| } | ||
| } | ||
|
|
||
| func buildIgnoreRulesFromMatchExclusion(e grypeDB.VulnerabilityMatchExclusion) []match.IgnoreRule { | ||
| var ignoreRules []match.IgnoreRule | ||
|
|
||
| if len(e.Constraints) == 0 { | ||
| ignoreRules = append(ignoreRules, match.IgnoreRule{Vulnerability: e.ID}) | ||
| return ignoreRules | ||
| } | ||
|
|
||
| for _, c := range e.Constraints { | ||
| ignoreRules = append(ignoreRules, match.IgnoreRule{ | ||
| Vulnerability: e.ID, | ||
| Namespace: c.Vulnerability.Namespace, | ||
| FixState: string(c.Vulnerability.FixState), | ||
| Package: match.IgnoreRulePackage{ | ||
| Name: c.Package.Name, | ||
| Language: c.Package.Language, | ||
| Type: c.Package.Type, | ||
| Location: c.Package.Location, | ||
| Version: c.Package.Version, | ||
| }, | ||
| }) | ||
| } | ||
|
|
||
| return ignoreRules | ||
| } | ||
|
|
||
| func (pr *MatchExclusionProvider) GetRules(vulnerabilityID string) ([]match.IgnoreRule, error) { | ||
| matchExclusions, err := pr.reader.GetVulnerabilityMatchExclusion(vulnerabilityID) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("match exclusion provider failed to fetch records for vulnerability id='%s': %w", vulnerabilityID, err) | ||
| } | ||
|
|
||
| var ignoreRules []match.IgnoreRule | ||
|
|
||
| for _, e := range matchExclusions { | ||
| rules := buildIgnoreRulesFromMatchExclusion(e) | ||
| ignoreRules = append(ignoreRules, rules...) | ||
| } | ||
|
|
||
| return ignoreRules, nil | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,4 +15,5 @@ type StoreWriter interface { | |
| IDWriter | ||
| VulnerabilityStoreWriter | ||
| VulnerabilityMetadataStoreWriter | ||
| Close() | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,4 +15,5 @@ type StoreWriter interface { | |
| IDWriter | ||
| VulnerabilityStoreWriter | ||
| VulnerabilityMetadataStoreWriter | ||
| Close() | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.