Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: Security TRG 8 UPDATED #681

Merged
merged 86 commits into from
Mar 8, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
2f130c7
Security TRG
klaudiaZF Feb 9, 2024
5eb555c
Merge pull request #1 from klaudiaZF/klaudiaZF-patch-1
klaudiaZF Feb 9, 2024
84c2195
Delete docs/release/trg-8 directory
klaudiaZF Feb 16, 2024
1c09c63
Security TRG 8
klaudiaZF Feb 16, 2024
47b4add
Merge branch 'eclipse-tractusx:main' into main
klaudiaZF Feb 16, 2024
34e362d
Delete docs/release/trg-8/trg-8-07.md
klaudiaZF Feb 19, 2024
513590a
Update trg-8-00.md
klaudiaZF Feb 22, 2024
68b4111
Update trg-8-01.md
klaudiaZF Feb 22, 2024
2ed2681
Delete docs/release/trg-8/trg-8-05.md
klaudiaZF Feb 22, 2024
0eb1838
Update trg-8-02.md
klaudiaZF Feb 22, 2024
39aacf3
Update trg-8-03.md
klaudiaZF Feb 22, 2024
35c06c7
Update trg-8-04.md
klaudiaZF Feb 22, 2024
a073b04
Update and rename trg-8-06.md to trg-8-05.md
klaudiaZF Feb 22, 2024
8bfa754
Update trg-8-00.md
klaudiaZF Feb 23, 2024
07e1014
Update trg-8-01.md
klaudiaZF Feb 26, 2024
3e779d4
Update trg-8-01.md
klaudiaZF Feb 26, 2024
58905f7
Update trg-8-01.md
klaudiaZF Feb 26, 2024
6540bbc
Update trg-8-02.md
klaudiaZF Feb 26, 2024
076e242
Update trg-8-04.md
klaudiaZF Feb 26, 2024
7778f1f
Update trg-8-03.md
klaudiaZF Feb 26, 2024
06f3f7c
Update trg-8-05.md
klaudiaZF Feb 26, 2024
7a1c81a
Update trg-8-03.md
klaudiaZF Feb 26, 2024
369cd06
Update trg-8-00.md
klaudiaZF Feb 26, 2024
aa22049
Update trg-8-00.md
klaudiaZF Feb 26, 2024
92a0108
Update trg-8-00.md
klaudiaZF Feb 26, 2024
a10ccb5
Update trg-8-00.md
klaudiaZF Feb 26, 2024
97c0118
Update trg-8-00.md
klaudiaZF Feb 29, 2024
51c3d81
Update trg-8-05.md
klaudiaZF Feb 29, 2024
c332531
Update trg-8-03.md
klaudiaZF Feb 29, 2024
6c75967
Update trg-8-05.md
klaudiaZF Feb 29, 2024
b44cd12
Update trg-8-03.md
klaudiaZF Feb 29, 2024
1ae0a02
Update trg-8-05.md
klaudiaZF Feb 29, 2024
358fc04
Update trg-8-03.md
klaudiaZF Feb 29, 2024
3a42ef3
Update trg-8-00.md
klaudiaZF Mar 1, 2024
7d1b949
Update trg-8-01.md
klaudiaZF Mar 1, 2024
59b8517
Update trg-8-03.md
klaudiaZF Mar 1, 2024
c64d434
Update trg-8-01.md
klaudiaZF Mar 1, 2024
491cb5c
Update trg-8-05.md
klaudiaZF Mar 1, 2024
8b8a49f
Update trg-8-05.md
klaudiaZF Mar 1, 2024
d613ace
docs: delete docs/release/trg-8/trg-8-00.md
scherersebastian Mar 1, 2024
1f66c56
docs: delete docs/release/trg-8/trg-8-02.md
scherersebastian Mar 1, 2024
c708ac0
docs: create trg-8-01.md
scherersebastian Mar 1, 2024
5ae780b
docs: add codeql workflow description
scherersebastian Mar 1, 2024
f36ad75
docs: better why for codeql
scherersebastian Mar 1, 2024
20f5a70
docs: small textual adjustments
scherersebastian Mar 1, 2024
1589198
docs: add trg-8-03 on kics
scherersebastian Mar 1, 2024
d36836a
docs: add paths ignore to codeql workflow
scherersebastian Mar 1, 2024
f88c5c8
docs: emphasize code
scherersebastian Mar 1, 2024
5217ab7
docs: update codeql version
scherersebastian Mar 1, 2024
2e425e3
docs: fail / exit strategy for kics
scherersebastian Mar 1, 2024
d835bc3
docs: correct cron job
scherersebastian Mar 2, 2024
23afbc7
docs: correct schedule
scherersebastian Mar 2, 2024
cbf234b
docs: correct grammar
scherersebastian Mar 2, 2024
7dba489
docs: correct grammar on comments
scherersebastian Mar 2, 2024
c4078ca
docs: codeql fail on error
scherersebastian Mar 2, 2024
ee7280a
docs: kics fail on error
scherersebastian Mar 2, 2024
25b350b
docs: take care of error severity findings
scherersebastian Mar 2, 2024
d7dbefb
docs: fix error and high severity findings
scherersebastian Mar 2, 2024
f951d20
docs: kics failure condition
scherersebastian Mar 4, 2024
f9bf291
docs: failure condition codeql
scherersebastian Mar 4, 2024
174bce0
docs: schedule kics
scherersebastian Mar 4, 2024
8d48374
docs: update trg-8-01.md
scherersebastian Mar 4, 2024
2c79c51
docs: update trg-8-03.md
scherersebastian Mar 4, 2024
d29c3d7
docs: update trg-8-01.md
scherersebastian Mar 4, 2024
a297b38
docs: add trivy draft trg
scherersebastian Mar 4, 2024
10f5ca4
docs: add gitguardian draft trg
scherersebastian Mar 4, 2024
291e35d
Delete docs/release/trg-8/trg-8-05.md
klaudiaZF Mar 5, 2024
393dc15
Delete docs/release/trg-8/trg-8-04.md
klaudiaZF Mar 5, 2024
75c15ea
Delete docs/release/trg-8/trg-8-03.md
klaudiaZF Mar 5, 2024
3923610
Delete docs/release/trg-8/trg-8-01.md
klaudiaZF Mar 5, 2024
fae42e0
Delete docs/release/trg-8/_category_.json
klaudiaZF Mar 5, 2024
5d1702b
adding caution
klaudiaZF Mar 7, 2024
b7043c9
Update trg-8-05.md
klaudiaZF Mar 7, 2024
9f96b14
Update trg-8-05.md
klaudiaZF Mar 7, 2024
ddbe6cd
Caution about QG
klaudiaZF Mar 7, 2024
1e0c6cf
Caution about QG
klaudiaZF Mar 7, 2024
8a149d3
Caution about QG
klaudiaZF Mar 7, 2024
8caf99c
grammar correction
klaudiaZF Mar 7, 2024
82a23a2
docs: update trg-8-01.md
scherersebastian Mar 7, 2024
810521c
docs: update trg-8-03.md
scherersebastian Mar 7, 2024
16e7b83
docs: update trg-8-05.md
scherersebastian Mar 7, 2024
c0d4b99
docs: update trg-8-04.md
scherersebastian Mar 7, 2024
d5bfa3b
docs: update trg-8-03.md
scherersebastian Mar 7, 2024
02b3c89
docs: wrong trg caution
scherersebastian Mar 7, 2024
4a8ab8a
docs: add statement about IP issues
scherersebastian Mar 7, 2024
52e39dc
docs: update trg-8-05.md
scherersebastian Mar 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/release/trg-8/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"label": "TRG 8 - Security"
}
66 changes: 66 additions & 0 deletions docs/release/trg-8/trg-8-00.md
scherersebastian marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
title: TRG 8.00 - Security Scanning Toolchain
---

| Status | Created | Post-History |
|--------|-------------|--------------------------------------|
| Active | 21-Feb-2024 | Initial release |

## Why

Our primary aim is to improve security and define best practices across the Tractus-X ecosystem.

## Description

A security scanning toolchain is a collection of tools and processes that are used to scan software applications for security vulnerabilities. These tools can be used at various stages of the software development lifecycle (**SDLC**), from development to deployment and beyond.

:::caution

To pass the quality gates, all **critical** and **high** security vulnerabilities **must be mitigated**.
evegufy marked this conversation as resolved.
Show resolved Hide resolved

:::

## Tools that we’re using

- ### SAST (Static Application Security Testing)

Tools analyze source code or compiled binaries to identify potential vulnerabilities

**Open-Source**: [CodeQL](/docs/release/trg-8/trg-8-01), [Snyk](/docs/release/trg-8/trg-8-06)

- ### SCA (Software Composition Analysis)

Tools examine the software components

**Open-Source**: [Snyk](/docs/release/trg-8/trg-8-06)

- ### DAST (Dynamic Application Security Testing)

Tools test the application in it is running state to identify vulnerabilities that may not be detected by SAST

**Open-Source**: [Owasp ZAP](/docs/release/trg-8/trg-8-05)

- ### IaC (Infrastructure as Code)

Tools that check the configuration files that define the infrastructure components of an application

**Open-Source**: [KICS](/docs/release/trg-8/trg-8-03), [Snyk](/docs/release/trg-8/trg-8-06)

- ### Secret Scanning

Tools designed to search for and identify sensitive information, known as secrets, within code repositiories
evegufy marked this conversation as resolved.
Show resolved Hide resolved

**Open-Source**: [GitGuardian](/docs/release/trg-8/trg-8-02)

- ### Container Scanner

Tools that scan the container images and the running containers

**Open-Source**: [Trivy](/docs/release/trg-8/trg-8-04), [Snyk](/docs/release/trg-8/trg-8-06)

:::tip

Security is not a one-time activity, but a continuous process that requires constant attention and improvement.
Even if you cannot perform a full **security assessment** for each product every release, you should at least follow basic security practices.

:::
135 changes: 135 additions & 0 deletions docs/release/trg-8/trg-8-01.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
---
title: TRG 8.01 - CodeQL
---

| Status | Created | Post-History |
|--------|-------------|--------------------------------------|
| Active | 21-Feb-2024 | Initial release |

## Why

Our primary aim is to improve security and define best practices across the Tractus-X ecosystem.

## Description

A security scanning toolchain is a collection of tools and processes that are used to scan software applications for security vulnerabilities. These tools can be used at various stages of the software development lifecycle (**SDLC**), from development to deployment and beyond.

evegufy marked this conversation as resolved.
Show resolved Hide resolved
### CodeQL

**CodeQL** serves as our core code analysis tool (**SAST**), providing deep code introspection for potential security vulnerabilities and other code quality concerns.
Below is a technical breakdown of how CodeQL integrates with our **CI/CD** process.

:::info

The CodeQL scan is triggered upon commits to the main branch, based on a CRON schedule set at 01:36 every Sunday, or when manually initiated.

:::

Given the range of languages CodeQL can analyze, the workflow leverages a matrix strategy to dynamically adjust runner settings based on the target language. It currently scans **Java**, **JavaScript**, **Python**, and **Ruby**, but this list is adjustable depending on the repository's dominant languages.

:::info

CodeQL supports a broader set of languages including 'cpp', 'csharp', 'go', 'swift', among others. Accordingly, adjustments should be made to the language matrix when different languages are in play.

:::

### The CodeQL analysis consists of several steps:

- **Repository Checkout**: The repository content is fetched using actions/checkout@v3.

- **CodeQL Initialization**: The github/codeql-action/init@v2 action initializes the CodeQL tools, setting the target languages and the desired query sets. CodeQL possesses an extensive collection of predefined queries, but developers can specify custom queries if necessary.

- **Auto-build**: The github/codeql-action/autobuild@v2 action attempts to build any compiled languages. This auto-build feature can occasionally face issues and may fail, especially with complex build processes or non-standard configurations. If an auto-build failure occurs, developers must manually configure the build process within the workflow. An example is provided in the workflow to guide this manual setup.

- **CodeQL Analysis**: Post build, CodeQL performs its analysis, examining the codebase for vulnerabilities and other concerns. Results are categorized based on the language of analysis.

In the provided CodeQL workflow, specific queries are used to enhance security analysis: +security-extended,security-and-quality. The + symbol ensures that these queries are added to the default set, allowing for a comprehensive security analysis. Developers should be aware of these configured queries as they focus on identifying a broad range of vulnerabilities, ensuring robust code security and quality.

```md
scherersebastian marked this conversation as resolved.
Show resolved Hide resolved
/********************************************************************************
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: ["main"]
pull_request:
# The branches below must be a subset of the branches above
branches: ["main"]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The best practice says its not mandatory to run a security workflow on every pull request but here it is the default. Should the best practice then not say it is recommended to run every workflow on each pull request? Or should this here be removed on default to fit more with the best practice?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nicoprow Trivy, KICS and CodeQL are different types of scanners, that's the reason why workflows might be different. I made changes, please review them and let me know if now it's more clear for you. Thank you in advance

schedule:
- cron: "36 1 * * 0"
workflow_dispatch:

jobs:
analyze:
name: Analyze
# Runner size impacts CodeQL analysis time. To learn more, please see:
# - https://gh.io/recommended-hardware-resources-for-running-codeql
# - https://gh.io/supported-runners-and-hardware-resources
# - https://gh.io/using-larger-runners
# Consider using larger runners for possible analysis time improvements.
runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
timeout-minutes: ${{ (matrix.language == 'swift' && 120) || 360 }}
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: ["java", "javascript", "python", "ruby"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby', 'swift' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
- name: Checkout repository
uses: actions/checkout@v3

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.

# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
queries: +security-extended,security-and-quality

# Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift).
# Automates dependency installation for Python, Ruby, and JavaScript, optimizing the CodeQL analysis setup.
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2

# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun

# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.

# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"
********************************************************************************/
```

25 changes: 25 additions & 0 deletions docs/release/trg-8/trg-8-02.md
scherersebastian marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
title: TRG 8.02 - GitGuardian
---

| Status | Created | Post-History |
|--------|-------------|--------------------------------------|
| Active | 21-Feb-2024 | Initial release |

## Why

Our primary aim is to improve security and define best practices across the Tractus-X ecosystem.

## Description

A security scanning toolchain is a collection of tools and processes that are used to scan software applications for security vulnerabilities. These tools can be used at various stages of the software development lifecycle (**SDLC**), from development to deployment and beyond.

### GitGuardian

**GitGuardian** is integrated via its GitHub App, enabling automated secret scanning of our codebase. Each pull request (PR) undergoes a scan. If a potential secret is detected, the commit's author receives an immediate email notification.

:::info

The email contains a temporary **link**, allowing the author to either **report** the detected secret or **mark it as a false positive**, streamlining the review process for software engineers.

:::
91 changes: 91 additions & 0 deletions docs/release/trg-8/trg-8-03.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
title: TRG 8.03 - KICS
---

| Status | Created | Post-History |
|--------|-------------|--------------------------------------|
| Active | 21-Feb-2024 | Initial release |

## Why

Our primary aim is to improve security and define best practices across the Tractus-X ecosystem.

## Description

A security scanning toolchain is a collection of tools and processes that are used to scan software applications for security vulnerabilities. These tools can be used at various stages of the software development lifecycle (**SDLC**), from development to deployment and beyond.

### KICS

**KICS** is an integral tool in our security workflow, specifically targeting infrastructure-as-code (IaC) vulnerabilities. Here's how we've integrated KICS into our process:

:::info

When a push is made to the main branch or once daily (based on a CRON schedule), excluding markdown and text files, the KICS scan is triggered. Additionally, a manual dispatch option is available for on-demand scans.

:::

The job runs on the latest Ubuntu and requires permissions for reading actions and content, as well as writing security events. Upon initiation, the repository is checked out using the actions/checkout@v3 action.

The primary action involves running the KICS scan, which leverages the checkmarx/[email protected]. The scan focuses on the root directory, and the results are outputted in the SARIF format, stored in the kicsResults/ directory.

:::info

KICS is configured to exit with a status code of 0, regardless of the scan results, unless there's a KICS engine error. Some paths and specific queries are excluded from the scan, and secret scanning is explicitly disabled.

:::

Subsequently, the SARIF file, which contains the KICS scan results, is uploaded using the github/codeql-action/upload-sarif@v2 action. This ensures that the findings are made available for review and further analysis in the GitHub environment, aiding engineers in addressing potential vulnerabilities effectively.

```md
/********************************************************************************
name: Run KICS scan and upload SARIF

on:
push:
branches: main
paths-ignore:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is an interesting one. Here we trying to decide whether its worth it to run the analysis at all on a push. In theory you could make more use of that in other workflows as well. CodeQL for example does not need to be run on changes to Helm charts or docs. Is that something that should be recommended to do in the best practices?

Or another take: Since the workflow will be executed on schedule anyway regardless of any change is defining exclusions like this unnecessary in the first place?

- "**/*.md"
- "**/*.txt"
schedule:
- cron: "0 0 * * *" # Once a day
workflow_dispatch:

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

steps:
- name: Checkout repo
uses: actions/checkout@v3

- name: Run KICS Scan with SARIF result
uses: checkmarx/[email protected]
with:
# Scanning directory .
path: "."
# When provided with a directory on output_path
# it will generate the specified reports file named 'results.{extension}'
# in this example it will generate: kicsResults/results.sarif
output_path: kicsResults/
output_formats: "sarif"
# If you want KICS to ignore the results and return exit status code 0 unless a KICS engine error happens
ignore_on_exit: results
# Exclude paths or files from scan
# exclude_paths: "terraform/gcp/big_data.tf,terraform/azure"
# Exclude accepted queries from the build
# exclude_queries: 0437633b-daa6-4bbc-8526-c0d2443b946e
# No secret scanning
disable_secrets: true

- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: kicsResults/results.sarif
********************************************************************************/
```

71 changes: 71 additions & 0 deletions docs/release/trg-8/trg-8-04.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
title: TRG 8.04 - Trivy
---

| Status | Created | Post-History |
|--------|-------------|--------------------------------------|
| Active | 21-Feb-2024 | Initial release |

## Why

Our primary aim is to improve security and define best practices across the Tractus-X ecosystem.

## Description

A security scanning toolchain is a collection of tools and processes that are used to scan software applications for security vulnerabilities. These tools can be used at various stages of the software development lifecycle (**SDLC**), from development to deployment and beyond.

### Trivy

Trivy stands as our container vulnerability scanner of choice, ensuring the security of our container images by targeting both OS-level and library dependencies. Here's a concise breakdown of the Trivy integration in our workflow:

:::info

The Trivy scan is initiated either on-demand through manual dispatch or based on a CRON schedule, executing once daily. The job is executed on the latest Ubuntu and requires specified permissions: reading actions and content and writing security events.

:::

The primary step involves the Trivy vulnerability scanner pulling the container image tractusx/irs-api:latest from Docker Hub. Before scanning, it's essential to ensure that the desired image on Docker Hub is correctly configured for the scan.

:::caution

We recommend always scanning the most recently published image to maintain updated security assessments. Utilizing the aquasecurity/[email protected], the scanner inspects the image for vulnerabilities of types os and library. Results are formatted as SARIF and stored in trivy-results.sarif.

:::

After the scan, results are then uploaded to the GitHub Security tab via the github/codeql-action/upload-sarif@v2 action, ensuring engineers can efficiently review and address any highlighted vulnerabilities.

```md
/********************************************************************************
name: "Run Trivy scan and upload SARIF"

on:
workflow_dispatch:
schedule:
- cron: "0 0 * * *" # Once a day

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

steps:
# Pull image from Docker Hub and run Trivy vulnerability scanner
- name: Run Trivy vulnerability scanner
uses: aquasecurity/[email protected]
with:
image-ref: "tractusx/irs-api:latest"
format: "sarif"
output: "trivy-results.sarif"
vuln-type: "os,library"

- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: "trivy-results.sarif"
********************************************************************************/
```

Loading
Loading