New Parser: Kiuwan SCA #10522

mwager · 2024-07-06T09:44:14Z

Re-opening as per comment: #10064 (comment)

See description and discussion in the other PR: #10064

…he UI display

… kiuwan-sca * 'kiuwan-sca' of github.com:mwager/django-DefectDojo: Update versions in application files Product Metrics: Performance Enhancements (DefectDojo#10059) String Based Filtering: Follow on for DefectDojo#10038 (DefectDojo#10050) update semgrep tests (DefectDojo#10058) Jira Webhook: Reorg logging and responses (DefectDojo#10049) Similar Findings: Create Toggle (DefectDojo#10047) Bump social-auth-app-django from 5.4.0 to 5.4.1 (DefectDojo#10026) Update versions in application files Update versions in application files Updated DryRun Security config (DefectDojo#10037) Filtering Performance: Add opt-in setting for converting to string ba… (DefectDojo#10038) Updates to semgrep parser (DefectDojo#10033) Update versions in application files

dryrunsecurity · 2024-07-15T14:24:32Z

DryRun Security Summary

The pull request includes various updates and improvements to the integration of the Kiuwan Scanner (both SAST and SCA) with the DefectDojo application, including documentation updates, the addition of a new parser for Kiuwan SCA scan results, and the inclusion of unit tests to ensure the robustness of the Kiuwan SCA parser.

Expand for full summary

Summary:

The code changes in this pull request cover various updates and improvements to the integration of the Kiuwan Scanner (both SAST and SCA) with the DefectDojo application. The changes include documentation updates, the addition of a new parser for Kiuwan SCA scan results, and the inclusion of unit tests to ensure the robustness of the Kiuwan SCA parser.

From an application security perspective, the key points to highlight are:

Documentation Updates: The changes to the documentation provide more detailed information on how to integrate Kiuwan SAST and SCA scans with DefectDojo, which helps improve the usability and security-related functionality of the application.
Kiuwan SCA Parser: The addition of the KiuwanSCAParser class and the associated unit tests ensure that the DefectDojo application can accurately process and ingest the results of Kiuwan SCA scans. This includes mapping the Kiuwan vulnerability data to the appropriate fields, deduplicating findings, and handling muted (false positive) findings.
Secure API Integration: The documentation and code changes demonstrate the use of secure API authentication (e.g., Basic Authentication with a token) when integrating with the Kiuwan REST API. This is an important security consideration to prevent unauthorized access and potential data leakage.
Secure Handling of Scan Results: The unit tests cover the proper handling of the Kiuwan SCA scan results, including input validation and sanitization when processing the JSON data. This helps to mitigate potential security issues, such as JSON injection or insecure deserialization.
Vulnerability Monitoring and Remediation: The inclusion of sample Kiuwan SCA scan results, which contain details about various vulnerabilities, demonstrates the organization's commitment to proactive vulnerability monitoring and remediation.

Overall, the changes in this pull request appear to be a positive contribution to the security and functionality of the DefectDojo application, as they improve the integration and processing of Kiuwan SAST and SCA scan results.

Files Changed:

docs/content/en/integrations/parsers/file/kiuwan.md: Documentation update for the Kiuwan Scanner (SAST) integration.
docs/content/en/integrations/parsers/file/kiuwan-sca.md: Documentation for the Kiuwan Scanner (SCA) integration.
dojo/settings/settings.dist.py: Addition of a new parser for the "Kiuwan SCA Scan" tool, including updates to the deduplication algorithm and hash code fields.
dojo/settings/.settings.dist.py.sha256sum: Update to the SHA-256 hash value for the dojo/settings/.settings.dist.py file, indicating a change to the configuration file.
unittests/scans/kiuwan-sca/kiuwan_sca_no_vuln.json: Addition of an empty JSON array, indicating a Kiuwan SCA scan with no detected vulnerabilities.
unittests/scans/kiuwan-sca/kiuwan_sca_many_vuln.json: Addition of a file containing details about various vulnerabilities detected by the Kiuwan SCA tool.
dojo/tools/kiuwan_sca/parser.py: Implementation of the KiuwanSCAParser class, which is responsible for parsing the Kiuwan SCA scan results.
unittests/tools/test_kiuwan_sca_parser.py: Unit tests for the KiuwanSCAParser class.
unittests/scans/kiuwan-sca/kiuwan_sca_two_vuln.json: Addition of a file containing details about two security vulnerabilities and one muted vulnerability.

Code Analysis

We ran 9 analyzers against 10 files and 0 analyzers had findings. 9 analyzers had no findings.

Riskiness

🟢 Risk threshold not exceeded.

View PR in the DryRun Dashboard.

* dev: Update dependency ruff from 0.5.0 to v0.5.1 (requirements-lint.txt) (DefectDojo#10521) Bump django-debug-toolbar from 4.4.2 to 4.4.4 (DefectDojo#10520) Bump boto3 from 1.34.138 to 1.34.139 (DefectDojo#10518) Bump psycopg[binary] from 3.1.19 to 3.2.1 (DefectDojo#10517) Bump packageurl-python from 0.15.1 to 0.15.2 (DefectDojo#10516)

github-actions · 2024-07-15T14:28:36Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

mwager · 2024-07-16T04:26:05Z

@kiblik Do you know why all these tests are failing? Is this expected to happen?

https://github.com/DefectDojo/django-DefectDojo/actions/runs/9941292602/job/27463545277?pr=10522

kiblik · 2024-07-16T07:14:52Z

@kiblik Do you know why all these tests are failing? Is this expected to happen?

https://github.com/DefectDojo/django-DefectDojo/actions/runs/9941292602/job/27463545277?pr=10522

I know :)

django-DefectDojo/dojo/settings/settings.dist.py

Lines 7 to 11 in 828d1e7

    
           ######################################################################################################### 
        
           # If as a developer of a new feature, you need to perform an update of file 'settings.dist.py',         # 
        
           # after the change, calculate the checksum and store it related file by calling the following command:  # 
        
           # $ sha256sum settings.dist.py | cut -d ' ' -f1 > .settings.dist.py.sha256sum                           # 
        
           #########################################################################################################

… kiuwan-sca # By dependabot[bot] (13) and others # Via GitHub * 'kiuwan-sca' of github.com:mwager/django-DefectDojo: (39 commits) Deprecate Python-jose and migrate okta to python_social_auth (DefectDojo#10117) fix: dockerfile warnings (DefectDojo#10505) Ruff: Add and fix Q000 (DefectDojo#10095) Fix(django): Upgrade of 4.2 (DefectDojo#10553) fix(deps): build python psycopg3 dependency instead of use the pre-build binary (DefectDojo#10491) Bump coverage from 7.5.4 to 7.6.0 (DefectDojo#10560) Bump asteval from 1.0.0 to 1.0.1 (DefectDojo#10561) Bump djangorestframework from 3.14.0 to 3.15.2 (DefectDojo#10431) Bump boto3 from 1.34.142 to 1.34.143 (DefectDojo#10558) Bump django-debug-toolbar from 4.4.5 to 4.4.6 (DefectDojo#10557) Bump boto3 from 1.34.141 to 1.34.142 (DefectDojo#10551) Bump packageurl-python from 0.15.2 to 0.15.3 (DefectDojo#10541) Bump boto3 from 1.34.140 to 1.34.141 (DefectDojo#10542) Update helm lock file Update versions in application files Update versions in application files API: Convert get_filterset calls to get_queryset (DefectDojo#10543) Bump django-debug-toolbar from 4.4.4 to 4.4.5 (DefectDojo#10527) Fix ruff Ruff fix ... # Conflicts: # dojo/settings/.settings.dist.py.sha256sum

github-actions · 2024-07-16T10:36:10Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

mwager · 2024-07-16T10:37:25Z

@kiblik Do you know why all these tests are failing? Is this expected to happen?
https://github.com/DefectDojo/django-DefectDojo/actions/runs/9941292602/job/27463545277?pr=10522

I know :)

django-DefectDojo/dojo/settings/settings.dist.py

Lines 7 to 11 in 828d1e7

#########################################################################################################

# If as a developer of a new feature, you need to perform an update of file 'settings.dist.py', #

# after the change, calculate the checksum and store it related file by calling the following command: #

# $ sha256sum settings.dist.py | cut -d ' ' -f1 > .settings.dist.py.sha256sum #

#########################################################################################################

Thank you!😅

github-actions · 2024-07-16T10:39:12Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

mwager · 2024-07-16T14:31:34Z

Still these errors even though I added the sha256 sum, did I miss something?

https://github.com/DefectDojo/django-DefectDojo/actions/runs/9955319925/job/27511988262?pr=10522

kiblik · 2024-07-16T14:51:57Z

Still these errors even though I added the sha256 sum, did I miss something?

https://github.com/DefectDojo/django-DefectDojo/actions/runs/9955319925/job/27511988262?pr=10522

sha256 needs to be calculated based latest commit (to be more precise, based on changes from the latest commit where settings.dist.py was edited). I see that you did the calculation but afterward, you merged data from dev branch which contained an edit of settings.dist.py. So you need to do recalculation again.

mwager · 2024-07-18T05:53:54Z

There are e2e test issues, looks like typical async issues:
https://github.com/DefectDojo/django-DefectDojo/actions/runs/9959370422/job/27517136351?pr=10522#step:9:45

and also helm download issues:
https://github.com/DefectDojo/django-DefectDojo/actions/runs/9959370404/job/27516840500?pr=10522

What could I do? Maybe trigger a re-run?

mwager · 2024-07-29T08:34:46Z

@mtesauro as I already re-created this PR (see description above) I would like to avoid doing it again. This PR was reviewed already by 2 persons and just cannot get merged because of strange issues with the pipelines.

What could I do to help speed things up?

mtesauro · 2024-08-01T17:06:24Z

@mwager Sorry, you've had unfortunate timing - we're deprecating MySQL and RabbitMQ which were sprinkled all through our tests so they have been extra flaky in July which hasn't been helped by contributors being on holiday.

We're targeting this one to be part of next weeks release of 2.37.0 👍

mtesauro

Approved

mwager added 28 commits April 25, 2024 10:50

Update kiuwan docs

0ac5a69

Add Kiuwan SCA test files

718d8b1

Add Kiuwan SCA Parser Unittest

c44d701

Add Kiuwan SCA parser implementation

54e50d9

Add more fields to the Kiuwan SCA parser

9b4e9ef

Update parser docs

1d8146d

Add test case for "muted" findings

a137892

Update finding title

ae533cd

Minor cleanupo

4374447

Update hashing logic

ab6ad9f

Remove print statement

9660329

Add optional epss support

d07b9d5

Add epss unit test

cd37fe9

Add custom deduplication logic as default static is not enough for SCA

274fb16

Remove cve as it is not allowed for deduplication

9345115

Set finding title to component name as this makes more sense within t…

fb052cc

…he UI display

Merge branch 'DefectDojo:master' into kiuwan-sca

84ec7f7

Fix lint warnings

690e6cd

Fix another lint warning

13a6bf5

Merge branch 'dev' into kiuwan-sca

fc44d4f

Fix lint error

eb1711d

fix lint errors

dbb0ed9

fix lint errors

3bbf308

fix lint errors

70bdd19

Merge branch 'dev' into kiuwan-sca

a137ba3

Fix lint

589b249

Fix lint

5c8a080

github-actions bot added docker settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR labels Jul 6, 2024

github-actions bot removed apiv2 integration_tests ui helm labels Jul 15, 2024

mwager added 2 commits July 15, 2024 16:25

Merge branch 'dev' into kiuwan-sca

0fd219c

github-actions bot removed the conflicts-detected label Jul 15, 2024

mwager added 2 commits July 16, 2024 12:33

chore: add sha256sum of settings.dist.py

7410770

github-actions bot added the conflicts-detected label Jul 16, 2024

Merge branch 'dev' into kiuwan-sca

e458c17

github-actions bot removed the conflicts-detected label Jul 16, 2024

chore: add sha256sum of settings.dist.py

c3fdb3b

mtesauro approved these changes Aug 1, 2024

View reviewed changes

Maffooch approved these changes Aug 1, 2024

View reviewed changes

blakeaowens approved these changes Aug 2, 2024

View reviewed changes

hblankenship approved these changes Aug 2, 2024

View reviewed changes

mtesauro merged commit 91b34af into DefectDojo:dev Aug 2, 2024
125 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Parser: Kiuwan SCA #10522

New Parser: Kiuwan SCA #10522

mwager commented Jul 6, 2024

dryrunsecurity bot commented Jul 15, 2024 •

edited

Loading

github-actions bot commented Jul 15, 2024

mwager commented Jul 16, 2024

kiblik commented Jul 16, 2024

github-actions bot commented Jul 16, 2024

mwager commented Jul 16, 2024

github-actions bot commented Jul 16, 2024

mwager commented Jul 16, 2024

kiblik commented Jul 16, 2024

mwager commented Jul 18, 2024

mwager commented Jul 29, 2024

mtesauro commented Aug 1, 2024

mtesauro left a comment

New Parser: Kiuwan SCA #10522

New Parser: Kiuwan SCA #10522

Conversation

mwager commented Jul 6, 2024

dryrunsecurity bot commented Jul 15, 2024 • edited Loading

DryRun Security Summary

Code Analysis

Riskiness

github-actions bot commented Jul 15, 2024

mwager commented Jul 16, 2024

kiblik commented Jul 16, 2024

github-actions bot commented Jul 16, 2024

mwager commented Jul 16, 2024

github-actions bot commented Jul 16, 2024

mwager commented Jul 16, 2024

kiblik commented Jul 16, 2024

mwager commented Jul 18, 2024

mwager commented Jul 29, 2024

mtesauro commented Aug 1, 2024

mtesauro left a comment

Choose a reason for hiding this comment

dryrunsecurity bot commented Jul 15, 2024 •

edited

Loading