Remove MySQL and RabbitMQ

[Maffooch]

Following the deprecation of MySQL and RabbitMQ (see discussion post here), it is now time for removal. The following places have been touched:

• Unit tests
  • Remove matrix values for testing mysql/rabbitmq
  • Remove the --profile flag from other tests using dc- helper scripts
• Simplify docker compose to remove the need for profiles
• Remove all docker profile environments
  • This includes setting default values in the compose file for all Postgres/redis related env vars
• Clean up the docs to remove suggestions that mysql/rabbitmq are supported/configurable
• Remove mysql/rabbitmq traces from the helm chart
• Add breaking changes to 2.37.0 release notes

[sc-7056]

[dryrunsecurity]

DryRun Security Summary

The pull request includes a wide range of updates to the DefectDojo application's infrastructure, focusing on improving the development, testing, and deployment processes, while ensuring security best practices are followed.

Expand for full summary

Summary:

The code changes in this pull request cover a wide range of updates to the DefectDojo application's infrastructure, including GitHub workflows, Docker Compose configurations, and various utility scripts. The changes are primarily focused on improving the application's development, testing, and deployment processes, with a focus on simplifying the configuration, optimizing the test environments, and aligning with the latest Docker Compose functionality.

From an application security perspective, the changes do not introduce any obvious security concerns. The updates appear to be well-designed and follow security best practices, such as the removal of unnecessary dependencies, the use of environment variables for configuration, and the implementation of robust error handling and input validation. However, it's important to review the entire codebase and configuration to ensure that there are no potential security vulnerabilities or misconfigurations that could be introduced.

Files Changed:

1. GitHub Workflows:

   • .github/workflows/fetch-oas.yml: Updates the workflow for fetching OpenAPI Specifications (OAS) from a running DefectDojo instance.
   • .github/workflows/integration-tests.yml: Simplifies the workflow for running integration tests.
   • .github/workflows/k8s-tests.yml: Updates the workflow for deploying the application on a Kubernetes cluster.
   • .github/workflows/rest-framework-tests.yml: Simplifies the workflow for running unit tests on the Django Rest Framework.

2. Docker Compose Configuration:

   • docker-compose.yml: Simplifies the configuration by removing unnecessary services and using environment variables for configuration.
   • Various docker-compose.override.*.yml files: Updates the configuration for different environments, such as development, unit tests, and integration tests.

3. Utility Scripts:

   • dc-down.sh, dc-stop.sh, dc-integration-tests.sh, dc-unittest.sh, dc-up-d.sh, dc-up.sh: Updates the scripts responsible for managing the Docker Compose environment.
   • docker/setEnv.sh: Improves the management of different application environments.
   • docker/entrypoint-unit-tests-devDocker.sh, docker/entrypoint-unit-tests.sh: Updates the scripts responsible for running unit tests in the Docker environment.

4. Other Files:

   • Dockerfile.django-debian, Dockerfile.django-alpine, Dockerfile.nginx-alpine, Dockerfile.nginx-debian: Updates the Dockerfiles for the application's components.
   • NOTICE: Updates the NOTICE file with the latest third-party dependencies and their licenses.

Code Analysis

We ran 9 analyzers against 30 files and 0 analyzers had findings. 9 analyzers had no findings.

Riskiness

🟢 Risk threshold not exceeded.

View PR in the DryRun Dashboard.

[Maffooch]

Converting to draft until I get to the bottom of rest test framework failures

[mtesauro]

Approved

[hblankenship]

I note that there is a comment from gitbot regarding the following:

Check warning on line 461 in helm/defectdojo/values.yaml

GitHub Actions
/ Lint chart

461:12 [truthy] truthy value should be one of [false, true]

[cneill]

A few comments on this one, and a couple files that weren't totally purged of MySQL/RabbitMQ references yet:

• We can yank these lines ignoring MySQL/RabbitMQ out of renovate.json now
• This gory bit of dojo/metrics/utils.py
• This bit in settings.dist.py
• The mysqlclient blob in NOTICE

There are some other comments about MySQL-specific behavior scattered in some other files but we can worry about that later - I don't think it will hurt anything to leave them alone for now.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[Maffooch]

• This gory bit of dojo/metrics/utils.py
• This bit in settings.dist.py

I would really rather not touch these to be honest 😅 @cneill

[cneill]

• This gory bit of dojo/metrics/utils.py
• This bit in settings.dist.py

I would really rather not touch these to be honest 😅 @cneill

No problem, just thought I'd mention them since ~everything else is cleaned up and we might forget about them. But we're probably already doing enough with this PR.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
3 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud