Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ add support for Nuget ad-hoc commands (add/install) in Pinned Dependency checks #2779

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
1476084
add nuget pinned dependency checks
balteravishay Mar 22, 2023
7e37e4b
checks.yaml
balteravishay Mar 22, 2023
6b5be75
✨ GitLab: Security Policy check (#2754)
raghavkaul Mar 16, 2023
3875dfa
:seedling: Bump gocloud.dev from 0.26.0 to 0.29.0 (#2722)
dependabot[bot] Mar 17, 2023
c6ff0d4
:seedling: Bump github/codeql-action from 2.2.6 to 2.2.7
dependabot[bot] Mar 17, 2023
59fbd83
Remove unused code from changeset creation (#2776)
azeemshaikh38 Mar 21, 2023
a7911e4
:bug: Pass proper commit depth to github checkrun handler. (#2777)
spencerschrock Mar 21, 2023
e7c6f18
pr fixes
balteravishay Mar 26, 2023
41bc7b2
✨ Support for GitHub's internal integration (#2773)
laurentsimon Mar 22, 2023
2411511
🐛 Add tie breaker when sorting changesets by RevisionID in tests. (#2…
spencerschrock Mar 23, 2023
9a729fe
:seedling: enable fuzzing check in cron. (#2780)
spencerschrock Mar 23, 2023
e248713
:seedling: Bump tj-actions/changed-files from 35.7.0 to 35.7.6 (#2782)
dependabot[bot] Mar 23, 2023
c7dc491
:seedling: Bump actions/checkout from 3.3.0 to 3.4.0 (#2767)
dependabot[bot] Mar 23, 2023
596ba44
🌱 Bump golangci-lint and fix configuration file. (#2783)
spencerschrock Mar 23, 2023
fabc4bc
:seedling: Bump github.com/onsi/ginkgo/v2 from 2.9.0 to 2.9.2 in /too…
dependabot[bot] Mar 24, 2023
d10ecba
:seedling: Bump github/codeql-action from 2.2.7 to 2.2.8
dependabot[bot] Mar 26, 2023
aabecdf
:seedling: Bump actions/dependency-review-action from 3.0.3 to 3.0.4 …
dependabot[bot] Mar 27, 2023
83a9f5d
:bug: Restore upload of existing raw result Big Query data (#2795)
spencerschrock Mar 27, 2023
269c638
:seedling: Bump tj-actions/changed-files from 35.7.6 to 35.7.7 (#2797)
dependabot[bot] Mar 28, 2023
f17e8e6
🌱 Restore API quota metrics for the weekly cron job. (#2799)
spencerschrock Mar 28, 2023
19feebc
:seedling: Bump github.com/golangci/golangci-lint in /tools (#2794)
dependabot[bot] Mar 28, 2023
18e647c
:seedling: Bump google.golang.org/protobuf in /tools (#2759)
dependabot[bot] Mar 28, 2023
b23bb93
:seedling: Bump golang.org/x/tools from 0.6.0 to 0.7.0 (#2769)
dependabot[bot] Mar 29, 2023
186c1a1
:seedling: Bump github.com/xanzy/go-gitlab from 0.78.0 to 0.81.0 (#2737)
dependabot[bot] Mar 29, 2023
c651743
:seedling: Bump actions/stale from 6.0.1 to 8.0.0 (#2793)
dependabot[bot] Mar 29, 2023
0a6a861
:seedling: Bump actions/setup-go from 3.5.0 to 4.0.0 (#2757)
dependabot[bot] Mar 29, 2023
12d0799
:seedling: Bump goreleaser/goreleaser-action from 4.1.0 to 4.2.0 (#2628)
dependabot[bot] Mar 29, 2023
bf7d683
:seedling: Bump github.com/google/osv-scanner (#2803)
dependabot[bot] Mar 29, 2023
500c15c
:seedling: Bump github.com/bradleyfalzon/ghinstallation/v2 (#2805)
dependabot[bot] Mar 30, 2023
ce8e18e
:seedling: Bump cloud.google.com/go/pubsub from 1.28.0 to 1.30.0 (#2804)
dependabot[bot] Mar 30, 2023
92036c0
:seedling: Bump github.com/goreleaser/goreleaser in /tools (#2770)
dependabot[bot] Mar 30, 2023
9889d62
:seedling: Bump actions/checkout from 3.4.0 to 3.5.0 (#2800)
dependabot[bot] Mar 30, 2023
aeae1a7
:seedling: Bump github/codeql-action from 2.2.8 to 2.2.9 (#2802)
dependabot[bot] Mar 30, 2023
edf2668
:seedling: Bump tj-actions/changed-files from 35.7.7 to 35.7.8 (#2801)
dependabot[bot] Mar 30, 2023
08949b0
:seedling: Bump github.com/moby/buildkit from 0.11.4 to 0.11.5 (#2809)
dependabot[bot] Mar 30, 2023
81523f5
:seedling: Bump ossf/scorecard-action from 2.1.2 to 2.1.3 (#2806)
dependabot[bot] Mar 31, 2023
a10376a
:seedling: Bump github.com/google/osv-scanner from 1.3.0 to 1.3.1 (#2…
dependabot[bot] Mar 31, 2023
7e9dd14
:seedling: Bump github.com/onsi/gomega from 1.27.0 to 1.27.6 (#2807)
dependabot[bot] Mar 31, 2023
7ddfdeb
:seedling: Bump cloud.google.com/go/bigquery from 1.48.0 to 1.49.0
dependabot[bot] Apr 2, 2023
d2d8c11
:seedling: Bump github.com/go-logr/logr from 1.2.3 to 1.2.4 (#2813)
dependabot[bot] Apr 3, 2023
fc71e8c
:seedling: Bump cloud.google.com/go/bigquery from 1.49.0 to 1.50.0 (#…
dependabot[bot] Apr 4, 2023
afff24f
:seedling: Bump step-security/harden-runner from 2.2.1 to 2.3.0 (#2823)
dependabot[bot] Apr 5, 2023
56841e0
:seedling: Bump github.com/docker/docker in /tools (#2825)
dependabot[bot] Apr 5, 2023
040f114
:seedling: Bump github/codeql-action from 2.2.9 to 2.2.11 (#2836)
dependabot[bot] Apr 7, 2023
0fd7ccd
:seedling: Bump tj-actions/changed-files from 35.7.8 to 35.7.12
dependabot[bot] Apr 9, 2023
4b33639
:seedling: Bump sigstore/cosign-installer from 3.0.1 to 3.0.2 (#2842)
dependabot[bot] Apr 10, 2023
a903241
:seedling: Bump github.com/xeipuuv/gojsonschema
dependabot[bot] Apr 10, 2023
2e2878f
:seedling: Unit tests for checker result and request (#2844)
naveensrinivasan Apr 10, 2023
c93b2eb
:sparkles: Consider haskell-actions/hlint-scan a code scanning action…
chungyc Apr 11, 2023
de66f95
:seedling: Bump github.com/bradleyfalzon/ghinstallation/v2 (#2847)
dependabot[bot] Apr 11, 2023
de43884
:seedling: Bump github.com/otiai10/copy from 1.9.0 to 1.10.0
dependabot[bot] Apr 11, 2023
84d9b56
:seedling: Bump github.com/goreleaser/goreleaser in /tools
dependabot[bot] Apr 11, 2023
0a3467c
🌱 Add instructions to test cron controller + worker locally (#2817)
spencerschrock Apr 11, 2023
eecb7ff
:seedling: Bump golang.org/x/tools from 0.7.0 to 0.8.0 (#2855)
dependabot[bot] Apr 12, 2023
8ba46d5
:seedling: Bump codecov/codecov-action from 3.1.0 to 3.1.2
dependabot[bot] Apr 12, 2023
413a1d7
:book: Fix broken links. (#2858)
chungyc Apr 12, 2023
aecedc7
:sparkles: Detect fuzzing in Haskell by the presence of property test…
chungyc Apr 12, 2023
ca089ae
:seedling: Unit tests for attestor policy (#2857)
naveensrinivasan Apr 12, 2023
888adf6
:seedling: Bump github.com/xanzy/go-gitlab from 0.81.0 to 0.82.0
dependabot[bot] Apr 12, 2023
f6fb843
✨ Use local files instead of search for SAST CodeQL check (#2839)
spencerschrock Apr 12, 2023
96dd113
.exe
balteravishay Apr 13, 2023
5ccdf0b
lint
balteravishay Apr 13, 2023
6bce00f
Merge branch 'main' into avbalter/support-nuget-unpinned-dependencies
balteravishay Apr 14, 2023
40438e4
pr comments
balteravishay Apr 14, 2023
3a0da40
Merge branch 'main' into avbalter/support-nuget-unpinned-dependencies
balteravishay Apr 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion checker/raw_result.go
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,10 @@ const (
DependencyUseTypeChocoCommand DependencyUseType = "chocoCommand"
// DependencyUseTypeNpmCommand is an npm command.
DependencyUseTypeNpmCommand DependencyUseType = "npmCommand"
// DependencyUseTypePipCommand is a pipp command.
// DependencyUseTypePipCommand is a pip command.
DependencyUseTypePipCommand DependencyUseType = "pipCommand"
// DependencyUseTypeNugetCommand is a nuget command.
DependencyUseTypeNugetCommand DependencyUseType = "nugetCommand"
)

// PinningDependenciesData represents pinned dependency data.
Expand Down
24 changes: 21 additions & 3 deletions checks/raw/pinned_dependencies_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ func TestGithubWorkflowPkgManagerPinning(t *testing.T) {
{
name: "npm packages without verification",
filename: "./testdata/.github/workflows/github-workflow-pkg-managers.yaml",
warns: 46,
warns: 49,
},
}
for _, tt := range tests {
Expand Down Expand Up @@ -819,6 +819,24 @@ func TestShellscriptInsecureDownloadsLineNumber(t *testing.T) {
endLine: 56,
t: checker.DependencyUseTypePipCommand,
},
{
snippet: "nuget install some-package",
startLine: 59,
endLine: 59,
t: checker.DependencyUseTypeNugetCommand,
},
{
snippet: "dotnet add package some-package",
startLine: 63,
endLine: 63,
t: checker.DependencyUseTypeNugetCommand,
},
{
snippet: "dotnet add SomeProject package some-package",
startLine: 64,
endLine: 64,
t: checker.DependencyUseTypeNugetCommand,
},
},
},
}
Expand Down Expand Up @@ -971,7 +989,7 @@ func TestDockerfileScriptDownload(t *testing.T) {
{
name: "pkg managers",
filename: "./testdata/Dockerfile-pkg-managers",
warns: 57,
warns: 60,
},
{
name: "download with some python",
Expand Down Expand Up @@ -1089,7 +1107,7 @@ func TestShellScriptDownload(t *testing.T) {
{
name: "pkg managers",
filename: "./testdata/script-pkg-managers",
warns: 53,
warns: 56,
},
{
name: "invalid shell script",
Expand Down
110 changes: 108 additions & 2 deletions checks/raw/shell_download_validate.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,9 @@ var (
pythonInterpreters = []string{"python", "python3", "python2.7"}
shellInterpreters = append([]string{"exec", "su"}, shellNames...)
otherInterpreters = []string{"perl", "ruby", "php", "node", "nodejs", "java"}
interpreters = append(otherInterpreters,
append(shellInterpreters, append(shellNames, pythonInterpreters...)...)...)
dotnetInterpreters = []string{"dotnet", "nuget"}
interpreters = append(dotnetInterpreters, append(otherInterpreters,
append(shellInterpreters, append(shellNames, pythonInterpreters...)...)...)...)
)

// Note: aws is handled separately because it uses different
Expand Down Expand Up @@ -696,6 +697,93 @@ func isChocoUnpinnedDownload(cmd []string) bool {
return true
}

func isUnpinnedNugetCliInstall(cmd []string) bool {
// looking for command of type nuget install ...
if len(cmd) < 2 {
return false
}

// Search for nuget commands.
if !isBinaryName("nuget", cmd[0]) && !isBinaryName("nuget.exe", cmd[0]) {
return false
}

// Search for install commands.
if !strings.EqualFold(cmd[1], "install") {
return false
}

// Assume installing a project with PackageReference (with versions)
// or packages.config at the root of command
if len(cmd) == 2 {
return false
}

// Assume that the script is installing from a packages.config file (with versions)
// package.config schema has required version field
// https://learn.microsoft.com/en-us/nuget/reference/packages-config#schema
// and Nuget follows Semantic Versioning 2.0.0 (versions are immutable)
// https://learn.microsoft.com/en-us/nuget/concepts/package-versioning#semantic-versioning-200
if strings.HasSuffix(cmd[2], "packages.config") {
laurentsimon marked this conversation as resolved.
Show resolved Hide resolved
laurentsimon marked this conversation as resolved.
Show resolved Hide resolved
return false
}

unpinnedDependency := true
for i := 2; i < len(cmd); i++ {
// look for version flag
if strings.EqualFold(cmd[i], "-Version") {
unpinnedDependency = false
break
}
}

return unpinnedDependency
}

func isUnpinnedDotNetCliInstall(cmd []string) bool {
// Search for command of type dotnet add <PROJECT> package <PACKAGE_NAME>
if len(cmd) < 4 {
return false
}
// Search for dotnet commands.
if !isBinaryName("dotnet", cmd[0]) && !isBinaryName("dotnet.exe", cmd[0]) {
return false
}

// Search for add commands.
if !strings.EqualFold(cmd[1], "add") {
return false
}

// Search for package commands (can be either the second or the third word)
if !(strings.EqualFold(cmd[2], "package") || strings.EqualFold(cmd[3], "package")) {
return false
}

unpinnedDependency := true
for i := 3; i < len(cmd); i++ {
// look for version flag
// https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-add-package
if strings.EqualFold(cmd[i], "-v") || strings.EqualFold(cmd[i], "--version") {
unpinnedDependency = false
break
}
}
return unpinnedDependency
}

func isNugetUnpinnedDownload(cmd []string) bool {
if isUnpinnedDotNetCliInstall(cmd) {
return true
}

if isUnpinnedNugetCliInstall(cmd) {
return true
}

return false
}

func collectUnpinnedPakageManagerDownload(startLine, endLine uint, node syntax.Node,
cmd, pathfn string, r *checker.PinningDependenciesData,
) {
Expand Down Expand Up @@ -782,6 +870,24 @@ func collectUnpinnedPakageManagerDownload(startLine, endLine uint, node syntax.N

return
}

// Nuget install.
if isNugetUnpinnedDownload(c) {
r.Dependencies = append(r.Dependencies,
checker.Dependency{
Location: &checker.File{
Path: pathfn,
Type: finding.FileTypeSource,
Offset: startLine,
EndOffset: endLine,
Snippet: cmd,
},
Type: checker.DependencyUseTypeNugetCommand,
},
)

return
}
// TODO(laurent): add other package managers.
}

Expand Down
103 changes: 103 additions & 0 deletions checks/raw/shell_download_validate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,109 @@ func TestValidateShellFile(t *testing.T) {
}
}

func Test_isDotNetUnpinnedDownload(t *testing.T) {
type args struct {
cmd []string
}
tests := []struct {
name string
args args
want bool
}{
{
name: "nuget install",
args: args{
cmd: []string{"nuget", "install", "Newtonsoft.Json"},
},
want: true,
},
{
name: "nuget restore",
args: args{
cmd: []string{"nuget", "restore"},
},
want: false,
},
{
name: "nuget install with -Version",
args: args{
cmd: []string{"nuget", "install", "Newtonsoft.Json", "-Version", "2"},
},
want: false,
},
{
name: "nuget install with packages.config",
args: args{
cmd: []string{"nuget", "install", "config\\packages.config"},
},
want: false,
},
{
name: "dotnet add",
args: args{
cmd: []string{"dotnet", "add", "package", "Newtonsoft.Json"},
},
want: true,
},
{
name: "dotnet add to project",
args: args{
cmd: []string{"dotnet", "add", "project1", "package", "Newtonsoft.Json"},
},
want: true,
},
{
name: "dotnet add reference to project",
args: args{
cmd: []string{"dotnet", "add", "project1", "reference", "OtherProject"},
},
want: false,
},
{
name: "dotnet build",
args: args{
cmd: []string{"dotnet", "build"},
},
want: false,
},
{
name: "dotnet add with -v",
args: args{
cmd: []string{"dotnet", "add", "package", "Newtonsoft.Json", "-v", "2.0"},
},
want: false,
},
{
name: "dotnet add to project with -v",
args: args{
cmd: []string{"dotnet", "add", "project1", "package", "Newtonsoft.Json", "-v", "2.0"},
},
want: false,
},
{
name: "dotnet add reference to project with -v",
args: args{
cmd: []string{"dotnet", "add", "project1", "reference", "Newtonsoft.Json", "-v", "2.0"},
},
want: false,
},
{
name: "dotnet add with --version",
args: args{
cmd: []string{"dotnet", "add", "package", "Newtonsoft.Json", "--version", "2.0"},
},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := isNugetUnpinnedDownload(tt.args.cmd); got != tt.want {
t.Errorf("isNugetUnpinnedDownload() = %v, want %v", got, tt.want)
}
})
}
}

func Test_isGoUnpinnedDownload(t *testing.T) {
type args struct {
cmd []string
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -161,4 +161,26 @@ jobs:
- name:
run: choco install --requirechecksums 'some-package'
- name:
run: choco install --require-checksums 'some-package'
run: choco install --require-checksums 'some-package'
- name:
run: nuget install 'some-package'
- name:
run: nuget restore
- name:
run: dotnet add package 'some-package'
- name:
run: dotnet add SomeProject package 'some-package'
- name:
run: nuget install 'some-package' -Version 1.2.3
- name:
run: nuget install packages.config
- name:
run: nuget install packages/packages.config
- name:
run: dotnet add package 'some-package' -v 1.2.3
- name:
run: dotnet build
- name:
run: dotnet add package 'some-package' --version 1.2.3
- name:
run: dotnet add SomeProject package 'some-package' --version 1.2.3
14 changes: 13 additions & 1 deletion checks/raw/testdata/Dockerfile-pkg-managers
Original file line number Diff line number Diff line change
Expand Up @@ -122,4 +122,16 @@ RUN choco install 'some-package'
RUN choco install 'some-other-package'
RUN choco install --requirechecksum 'some-package'
RUN choco install --requirechecksums 'some-package'
RUN choco install --require-checksums 'some-package'
RUN choco install --require-checksums 'some-package'


RUN nuget install some-package
RUN nuget restore
RUN nuget install some-package -Version 1.2.3
RUN nuget install packages.config
RUN dotnet add package some-package
RUN dotnet add SomeProject package some-package
RUN dotnet build
RUN dotnet add package some-package -v 1.2.3
RUN dotnet add package some-package --version 1.2.3
RUN dotnet add SomeProject package some-package --version 1.2.3
10 changes: 10 additions & 0 deletions checks/raw/testdata/script-pkg-managers
Original file line number Diff line number Diff line change
Expand Up @@ -123,3 +123,13 @@ choco install 'some-other-package'
choco install --requirechecksum 'some-package'
choco install --requirechecksums 'some-package'
choco install --require-checksums 'some-package'

nuget install some-package
nuget restore
nuget install some-package -Version 1.2.3
nuget install packages.config
dotnet add package some-package
dotnet add SomeProject package some-package
dotnet build
dotnet add package some-package -v 1.2.3
dotnet add package some-package --version 1.2.3
12 changes: 11 additions & 1 deletion checks/raw/testdata/shell-download-lines.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,14 @@ pip install --no-deps -e . git+https://github.com/username/repo.git
pip install --no-deps -e . git+https://github.com/username/repo.git@0123456789abcdef0123456789abcdef01234567#egg=package

python -m pip install --no-deps -e git+https://github.com/username/repo.git
python -m pip install --no-deps -e git+https://github.com/username/repo.git@0123456789abcdef0123456789abcdef01234567#egg=package
python -m pip install --no-deps -e git+https://github.com/username/repo.git@0123456789abcdef0123456789abcdef01234567#egg=package

nuget install some-package
nuget restore
nuget install some-package -Version 1.2.3
nuget install packages.config
dotnet add package some-package
dotnet add SomeProject package some-package
dotnet build
dotnet add package some-package -v 1.2.3
dotnet add package some-package --version 1.2.3
Loading