Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switch from incremental to full on config changes #36

Merged
merged 58 commits into from
Nov 2, 2023

Conversation

michaelkubiaczyk
Copy link
Collaborator

Changes

  • Tests
  • Documentation

michaelkubiaczyk and others added 30 commits September 5, 2023 21:49
* Initial in progress

* compiling but not yet functional

* Missed file

* updated checkmarxone step

* Working up to fetching a project then breaks

* Missed file

* Breaks when retrieving projects+proxy set

* Create project & run scan working, now polling

* Fixed polling

* added back the zipfile remove command

* Fixed polling again

* Generates and downloads PDF report

* Updated and working, prep for refactor

* Added compliance steps

* Cleanup, reporting, added groovy connector

* fixed groovy file

* checkmarxone to checkmarxOne

* checkmarxone to checkmarxOne

* split credentials (id+secret, apikey), renamed pullrequestname to branch, groovy fix

* Fixed filenames & yaml

* missed the metadata_generated.go

* added json to sarif conversion

* fix:type in new checkmarxone package

* fix:type in new checkmarxone package

* removed test logs, added temp error log for creds

* extra debugging to fix crash

* improved auth logging, fixed query parse issue

* fixed bug with group fetch when using oauth user

* CWE can be -1 if not defined, can't be uint

* Query also had CweID

* Disabled predicates-fetch in sarif generation

* Removing leftover info log message

* Better error handling

* fixed default preset configuration

* removing .bat files - sorry

* Cleanup per initial review

* refactoring per Gist, fixed project find, add apps

* small fix - sorry for commit noise while testing

* Fixing issues with incremental scans.

* removing maxretries

* Updated per PR feedback, further changes todo toda

* JSON Report changes and reporting cleanup

* removing .bat (again?)

* adding docs, groovy unit test, linter fixes

* Started adding tests maybe 15% covered

* fix(checkmarxOne): test cases for pkg and reporting

* fix(checkmarxOne):fix formatting

* feat(checkmarxone): update interface with missing method

* feat(checkmarxone):change runStep signature to be able to inject dependency

* feat(checkmarxone): add tests for step (wip)

* Adding a bit more coverage

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix code review

* feat(checkmarxOne): fix integration test PR

* adding scan-summary bug workaround, reportgen fail

* enforceThresholds fix when no results passed in

* fixed gap when preset empty in yaml & project conf

* fixed another gap in preset selection

* fix 0-result panic

* fail when no preset is set anywhere

* removed comment

* initial project-under-app support

* fixing sarif reportgen

* some cleanup of error messages

* post-merge test fixes

* revert previous upstream merge

* fix:formatting

* fix(checkmarxOne):yamllint too many blank lines

* fix(checkmarxOne):unit test

* fix(checkmarxOne):generated code

---------

Co-authored-by: thtri <[email protected]>
Co-authored-by: Thanh-Hai Trinh <[email protected]>
…ity] (SAP#4427)

* fix(deps): update module github.com/hashicorp/vault to v1.13.5 [security]

* fix(deps): update module github.com/Azure/azure-sdk-for-go/tree/sdk/storage/azblob to v0.4.1

* fix(deps): update module github.com/hashicorp/vault/sdk to v0.9.2
fix(deps): update module oras.land/oras-go to v1.2.3

* fix(deps): update module github.com/hashicorp/vault/sdk to v0.9.2-0.20230530190758-08ee474850e0
fix(deps): update module github.com/hashicorp/vault/sdk to v0.9.2-0.20230530190758-08ee474850e0

* replacing deprecated function

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Muhammadali Nazarov <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
* Add cnbBuild to buildExecute

* Error message change

* Make if check simpler

Co-authored-by: Alexander Link <[email protected]>

* Switch order of check

---------

Co-authored-by: Linda Siebert <[email protected]>
Co-authored-by: Linda Siebert <[email protected]>
Co-authored-by: Alexander Link <[email protected]>
Add option to opt out from helm template parsing

Co-authored-by: Linda Siebert <[email protected]>
Co-authored-by: Alexander Link <[email protected]>
…P#4559)

* fix(docs): fixing documentation for violating markdownlint rules.

* correct indent

* Apply suggestions from code review

* correct list

---------

Co-authored-by: Christopher Fenner <[email protected]>
* encrypt CPE - init

* fix

* disable encrypt on Jenkins

* get PIPER_pipelineEnv_SECRET from vault

* reuse artifactPrepareVersionOptions

* encrypt only with orchestrator.GitHubActions

* Workaround: orchestrators expect json

* add encryptedCPE flag

* remove JSON workaround

* throw error if stepConfigPassword is empty

* fix log messages

---------

Co-authored-by: Egor Balakin <[email protected]>
* Enable wdi5 autologin

By also providing the basic auth credential on the env vars wdi5_username and wdi5_password we enable the wdi5 autologin feature, where the user does not have to remap the credentials in their wdi5 configuration. See https://ui5-community.github.io/wdi5/#/authentication?id=credentials

* Add documentation

* Add wdi5 parameter

* Add tests
…sage of go >1.21 (SAP#4558)

* chore: ⬆️ upgrade to golang.org/x/mod/modfile v0.12.0 to enable the usage of go >1.21

* chore: run

---------

Co-authored-by: Jk1484 <[email protected]>
…ion issue (SAP#4565)

* removed quotation for detect8

* changed order of args

* chagned order
* Use new Golang implementation of tmsUpload step by default
…ons (SAP#4557)

* Allow running as different user on Kubernetes

Co-authored-by: Ralf Pannemans <[email protected]>
Co-authored-by: Johannes Dillmann <[email protected]>
Co-authored-by: Pavel Busko <[email protected]>

* infer securityContext from dockerOptions

Co-authored-by: Ralf Pannemans <[email protected]>
Co-authored-by: Pavel Busko <[email protected]>

* verify --user flag value

---------

Co-authored-by: Johannes Dillmann <[email protected]>
Co-authored-by: Ralf Pannemans <[email protected]>
Co-authored-by: Anil Keshav <[email protected]>
* made detect8 default

* amended tests to reflect detect 8

* amended

* amend

* amend

* amend tests

* 1

* 1

* tests-with-temp-changes-for-transition

* removed auto unmapping for detect7

* added-old-parameters-as-deprecated
Co-authored-by: Andrei Kireev <[email protected]>
…P#4563)

* refactor github package and use builder pattern for client

* switch to github package

* some renamings

* fix panic on uninitialized provider

* fix according to review comments

---------

Co-authored-by: Gulom Alimov <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
toolrecord file:
- drop the hardcoded default url
- use the more user-friendly project ID instead of the project token
* fix(deps): update github.com/bndr/gojenkins digest to 45fe314

* run go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Jordi van Liempt <[email protected]>
…AP#4447)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* fix(deps): update module golang.org/x/oauth2 to v0.12.0

* run go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
* vault retry check function

---------

Co-authored-by: I557621 <[email protected]>
…ntials config (SAP#4600)

* added logging if unauthorized for github

* refactored

* fixed log message & added logging github response

* deleted extra log

* refactored log message
anilkeshav27 and others added 28 commits September 28, 2023 11:31
… commit to trust enterprise github instances (SAP#4602)

* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test
…4569)

* add containerDockerfilePath param to multipleImages

* rename ContainerDockerfilePath param to DockerfilePath

* Fix trailing spaces

---------

Co-authored-by: Egor Balakin <[email protected]>
Co-authored-by: Vyacheslav Starostin <[email protected]>
…yer (SAP#4604)

forward sourceRepositoryCredentialsId from groovy to go layer in the same way how this is done for the targetRepositoryCredentialsId
…anges (SAP#4605)

* exporting generateConfig function and applying minor changes

* Added setConfigOptions to set configOptions variable.
Added possibility to set format output, json or yaml for now.

* Correcting mistake on cmd/getDefaults.go

Co-authored-by: Jordi van Liempt <[email protected]>

---------

Co-authored-by: Jordi van Liempt <[email protected]>
…AP#4595)

* added logic of fetching golang private packages for whitesource step and detectExecuteScan step

* changed logic of checking by config.PrivateModulesGitToken

* moved func prepareGolangPrivatePackages to golangBuild.go

* fix (gitOpsUpdateDeployment) add CA bundle options to plain clone and commit to trust enterprise github instances (SAP#4602)

* downloading ca cert bundle when added as config

* adding logging statements

* allowing bats test to handle ca cert

* adding info message

* hard coding file names

* including correct http client util bundle

* removing logging message not needed

* adding cert bundle to commit and push

* improving the condition to add ca cert in commit and push

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* feat(kanikoExecute): add dockerfilePath param to multipleImages (SAP#4569)

* add containerDockerfilePath param to multipleImages

* rename ContainerDockerfilePath param to DockerfilePath

* Fix trailing spaces

---------

Co-authored-by: Egor Balakin <[email protected]>
Co-authored-by: Vyacheslav Starostin <[email protected]>

* fix(helm): forward sourceRepositoryCredentialsId from groovy to go layer (SAP#4604)

forward sourceRepositoryCredentialsId from groovy to go layer in the same way how this is done for the targetRepositoryCredentialsId

* feat(config): exporting generateConfig function and applying minor changes (SAP#4605)

* exporting generateConfig function and applying minor changes

* Added setConfigOptions to set configOptions variable.
Added possibility to set format output, json or yaml for now.

* Correcting mistake on cmd/getDefaults.go

Co-authored-by: Jordi van Liempt <[email protected]>

---------

Co-authored-by: Jordi van Liempt <[email protected]>

* moved func prepareGolangPrivatePackages to pkg/golang

---------

Co-authored-by: Akramdzhon Azamov <[email protected]>
Co-authored-by: Andrei Kireev <[email protected]>
Co-authored-by: Anil Keshav <[email protected]>
Co-authored-by: Egor Balakin <[email protected]>
Co-authored-by: Egor Balakin <[email protected]>
Co-authored-by: Vyacheslav Starostin <[email protected]>
Co-authored-by: Marcus Holl <[email protected]>
Co-authored-by: Jk1484 <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
SAP#4579)

* enabling publish to only publish sub packages

* changing directory and then coming back to original after the publish runs

* searching the glob tar and npmrc in the current directory

* excluding build descriptor check and addtional target tool check

* changing the npm pack before publish to run only in sub packages

* removing commented code clean up

* adding the correct npm pack

* improve logging

* fix error handling and a bit style fix

* fix unit tests

* remove commented lines

* respecting build descriptor list when provided

* improve docu for the step param

* fixing linting issues

* improve docu

---------

Co-authored-by: Gulom Alimov <[email protected]>
Co-authored-by: Jordi van Liempt <[email protected]>
…be stashed/unstashed (SAP#4573)

* Add stash param to mavenBuild

* go generate

* Delete default stash group

---------

Co-authored-by: Anil Keshav <[email protected]>
* Add CertificateDownload func to certutils package

* Add customTlsCertificateLinks param for artifactPrepareVersion

* Add the possibility to provide custom certs for artifactPrepareVersion

* Update tests

* Return back build flags

* Return back build flags

* Update pkg/certutils/certutils.go

Apply suggestion from code review

Co-authored-by: Christopher Fenner <[email protected]>

---------

Co-authored-by: Christopher Fenner <[email protected]>
* structure vault logs

* Update client.go
…#4555)

* Update default_pipeline_environment.yml

Add filterRegex parameter to jmeter configuration. Set it to an empty string as generated from pipelinesyntax reference in jenkins

* Update testsPublishResults.groovy

Add the filterRegex param to the publishJmeterReport plugin in testPublishResults

* Update documentation to include filterRegex

---------

Co-authored-by: Anil Keshav <[email protected]>
* Adapting unit test to run for packages with namespace

* Handling classes with namespace

* support namespace obj for ATC
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.15.0 to 0.17.0.
- [Commits](golang/net@v0.15.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pload and export to node (SAP#4624)

* Provide additional log message on successful upload and export to node

---------

Co-authored-by: Oliver Feldmann <[email protected]>
…ity] (SAP#4550)

* fix(deps): update module github.com/hashicorp/vault to v1.14.3 [security]

* update to v1.14.1 instead due to Go version

* github.com/hashicorp/[email protected]

* go mod tidy

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: jliempt <>
Co-authored-by: Jordi van Liempt <[email protected]>
* fix(cxone): 1st scan failed due to 403 forbiden

* fix(cxone): 1st scan failed due to 403 forbiden

* fix(cxone): 1st scan failed due to 403 forbiden
…nks parameter (SAP#4631)

* Add conditions for customTlsCertificateLinks

* go generate
* Add new unified fields to Mend and Blackduck SARIF

* fmt project

---------

Co-authored-by: Dmitrii Pavlukhin <[email protected]>
* retry Vault request errors and add extensive logging

* fix sigsegv and change logging

* refining logs

* changing debug to info

---------

Co-authored-by: jliempt <>
Co-authored-by: Muhammadali Nazarov <[email protected]>
* Remove confusing comments in tmsClient.go
* Add warning message when cf native builds uses bg deployment

---------

Co-authored-by: Oliver Feldmann <[email protected]>
…hub (SAP#4652)

* fixed unzipping db sources without db dir

* fixed tests
@michaelkubiaczyk michaelkubiaczyk merged commit cca5388 into cxone-dev Nov 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.