Skip to content

Commit

Permalink
Automated integration test setup #2839
Browse files Browse the repository at this point in the history
- switched setup, start, stop parts for integration testing to bash scripts only
- improved integration test setup parts
- changed CI/CD build behaviors for "feature-" and "gha_feature-*"
  branches
- added documentation
  • Loading branch information
de-jcup committed Jan 26, 2024
1 parent 885dc4b commit 78e6348
Show file tree
Hide file tree
Showing 21 changed files with 540 additions and 49 deletions.
47 changes: 42 additions & 5 deletions .github/workflows/github-action-scan.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# SPDX-License-Identifier: MIT
name: Build SecHub GitHub Action
name: Build SecHub GHA (scan)

on: workflow_dispatch
on:
push:
branches:
- 'gha_*'

jobs:
build-scan:
Expand All @@ -20,13 +23,47 @@ jobs:
# which should be the default environment for the github actions runtime as well
uses: actions/setup-node@8f152de45cc393bb48ce5d89d36b731f54556e65

- name: Set up JDK 17
uses: actions/setup-java@387ac29b308b003ca37ba93a6cab5eb57c8f5f93
with:
java-version: 17
distribution: temurin

- name: Clean install
run: npm ci

- name: Build
run: npm run build --if-present

- name: Run tests
- name: Run unit tests
run: npm test



- name: Setup integration test data
id : version-selector
run: |
echo "sechub_server_version=1.4.1" >> "$GITHUB_ENV"
echo "sechub_server_port=8443" >> "$GITHUB_ENV"
echo "pds_version=1.4.0" >> "$GITHUB_ENV"
echo "pds_port=8444" >> "$GITHUB_ENV"
- name: Cache SecHub server download
# Cache V4 release: 13aacd865c20de90d75de3b17ebe84f7a17d57d2
uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2
with:
path: ~/runtime/server/${{ steps.version-selector.outputs.sechub_server_version }}/
key: ${{ runner.os }}-sechub-server-${{ steps.version-selector.outputs.sechub_server_version }}

- name: Cache PDS download
# Cache V4 release: 13aacd865c20de90d75de3b17ebe84f7a17d57d2
uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2
with:
path: ~/runtime/pds/${{ steps.version-selector.outputs.pds_version }}/
key: ${{ runner.os }}-pds-${{ steps.version-selector.outputs.pds_version }}
- name: Prepare for integration tests
run: ./__test__/integrationtest/01-start.sh $sechub_server_version $sechub_server_port $pds_version $pds_port

- name: Run integration tests
run: npm run integration-test

- name: Cleanup integration tests
run: ./__test__/integrationtest/05-stop.sh 8443 8444
4 changes: 3 additions & 1 deletion .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@ on:
# So this branch is only for github pages. See https://github.com/mercedes-benz/sechub/issues/481
# for details
- documentation
# We ignore everything where tag starts with v* - this is done by release build!
# ignore branches for github actions only - e.g. "gha_feature-12345-testfeature"
- gha_*
tags-ignore:
# We ignore everything where tag starts with v* - this is done by release build!
- v*
# enable manual triggering of workflow
workflow_dispatch:
Expand Down
36 changes: 30 additions & 6 deletions github-actions/scan/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ This GitHub action uses the SecHub cli to scan the repository for security issue

To be able to use this action you need a SecHub project. Check the https://mercedes-benz.github.io/sechub/[documentation] on how to set one up.

```yaml

[source,yaml]
----
- uses: mercedes-benz/sechub/github-actions/scan@72a27282da80952e6fadcef452c6a9085971c688
with:
# OPTIONAL: Path to sechub.json for manual configuration. If no value is set the input parameters will be used to create it for the scan.'
Expand Down Expand Up @@ -100,19 +100,43 @@ npm run test
==== Integration-Test
As a precondition to run the integration tests locally you have to

- start a new SecHub server in integration test mode
- execute `__test__/setup-integrationtest.sh`
- execute `__test__/01-start.sh $secHubServerVersion $sechubServerPortNr $pdsVersion $pdsPortNr`

When this has been done, you can execute this command multiple times:
After the script has been executed, you can execute integration tests multiple times via following command:

[source,npm]
----
npm run integration-test
----

To enable full debug output:
To enable full debug output in integration tests please execute following before running the integration tests:
[source,npm]
----
export SECHUB_DEBUG=true
----

==== Debug tests
The unit and also the integration tests are written with `jest` test framework.

===== Setup

**VSCodium**
Used extensions
- Test explorer
- Jest Test explorer
- Jest

In this setup the tests can be executed from sidebar and from links created inside editor.

But ... unfortunately, the Jest UI integration works only for npm script "test" which is normally not used for
integration tests (to handle them special).

If you want to debug an integration test, there is a workaround necessary:

- open `package.json` and look into section `scripts`
- switch `test-debug-all` to `test` and name former `test` script entry to `test-debug-not-integrationtest`
(but please do not push this - otherwise build will fail on integration test!)
- restart your VSCode/VSCodium instance (if integration tests are not listed in test explorer)
- debug the parts, fix it etc.
- switch `test` script entry to `test-debug-all` and `test-debug-not-integrationtest` to `test`.
- if necessary push fixes/changes to remote...
2 changes: 1 addition & 1 deletion github-actions/scan/__test__/init-scan.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ describe('initSecHubJson', function () {
const parameter = initSecHubJson('runtime/sechub.json','', [], []);

/* test */
expect(parameter).toBeNull();
expect(parameter).toEqual('runtime/sechub.json');
expect(createSecHubConfigJsonFile).toHaveBeenCalledTimes(1);
});
});
Expand Down
39 changes: 26 additions & 13 deletions github-actions/scan/__test__/integrationtest.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: MIT

import * as launcher from '../src/launcher';
import {IntegrationTestContext as IntegrationTestContext} from './integrationtest/testframework';
import * as shell from 'shelljs';
import { isDebug, debug, getInput } from '@actions/core';
import { info } from '@actions/core';
Expand All @@ -11,15 +12,31 @@ import { LaunchContext } from '../src/launcher';
import { create } from '@actions/artifact';
jest.mock('@actions/core');
jest.mock('@actions/artifact');

/*
* This is an integration test suite for github-action "scan".
* As precondition you have to start a local sechub server in integration test mode and execute "setup-integrationtest.sh".
* After this is done, you can execute these tests.
* This is an integration test suite for github-action "scan".
* As precondition you have to call "01-start.sh" (please look into script for an example and more details)
*
* After script has finished you can execute the integration tests via "npm run integration-test"
*
* At the end the servers can be stopped with "05-stop.sh" (please look into script for an example and more details)
* (This is an explanation to start the tests locally - the github action workflow "github-action-scan.yml" does it in exact same way for CI/CD)
*
*/

const sechub_debug = shell.env['SECHUB_DEBUG'];
const debug_enabled = sechub_debug=='true';

const integrationTestContext = new IntegrationTestContext();

integrationTestContext.serverVersion='1.4.0';
integrationTestContext.serverPort= 8443;
integrationTestContext.serverUserId='int-test_superadmin';
integrationTestContext.serverApiToken='int-test_superadmin-pwd';

integrationTestContext.finish();

const mockedInputMap = new Map();

beforeEach(() => {
jest.resetAllMocks();

Expand All @@ -34,7 +51,6 @@ beforeEach(() => {
return true;
});
}


(info as jest.Mock).mockImplementation((message) => {
console.log('gh-info: %s', message);
Expand All @@ -55,14 +71,13 @@ beforeEach(() => {
});


const mockedInputMap = new Map();

function initInputMap() {
mockedInputMap.clear();

mockedInputMap.set(input.PARAM_SECHUB_SERVER_URL, 'https://localhost:8443');
mockedInputMap.set(input.PARAM_SECHUB_USER, 'int-test_superadmin');
mockedInputMap.set(input.PARAM_API_TOKEN, 'int-test_superadmin-pwd');
mockedInputMap.set(input.PARAM_SECHUB_SERVER_URL, `https://localhost:${integrationTestContext.serverPort}`);
mockedInputMap.set(input.PARAM_SECHUB_USER, `${integrationTestContext.serverUserId}`);
mockedInputMap.set(input.PARAM_API_TOKEN, `${integrationTestContext.serverApiToken}`);
mockedInputMap.set(input.PARAM_PROJECT_NAME, 'test-project');

mockedInputMap.set(input.PARAM_CLIENT_VERSION, '1.2.0');
Expand All @@ -71,7 +86,6 @@ function initInputMap() {
mockedInputMap.set(input.PARAM_TRUST_ALL, 'true'); // self signed certificate in test...
}


describe('integrationtest', function () {
test('integrationtest 1', function () {

Expand All @@ -82,14 +96,13 @@ describe('integrationtest', function () {
const launchPromise = launcher.launch();

/* test */
assertExitCode(launchPromise, 0);
assertLastClientExitCode(launchPromise, 0);

});

});

async function assertExitCode(launchPromise: Promise<LaunchContext>, exitCode: number) {
async function assertLastClientExitCode(launchPromise: Promise<LaunchContext>, exitCode: number) {
const context = await launchPromise;

expect(context.lastClientExitCode).toEqual(exitCode);
}
147 changes: 147 additions & 0 deletions github-actions/scan/__test__/integrationtest/01-start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
#!/usr/bin/env bash
# SPDX-License-Identifier: MIT

# Purpose for this script:
# * Preparation for integration tests
#
# Details:
# - Calculates pathes
# - Downloads PDS and SecHub server version
# - removes former temp data automatically (e.g. old reports etc.)
# - Starts SecHubServer and PDS
# - Waits for the servers to be alive
# - After this a standard setup is done for SecHub to be able to communicate with PDS, project setup etc.
# - exit of script is done at the end - means synchronous execution
#
# Usage: 01-start.sh $secHubServerVersion $sechubServerPortNr $pdsVersion $pdsPortNr
# Example:
# ```
# cd $gitRoot/github-actions/scan
# ./01-start.sh 1.4.1 8443 1.4.0 8444
# ```
#
SERVER_VERSION=$1
SERVER_PORT=$2

PDS_SERVER_VERSION=$3
PDS_SERVER_PORT=$4

if [ "$SERVER_VERSION" = "" ]; then
echo "first argument not set - is used as server verion!"
exit 1
fi
if [ "$SERVER_PORT" = "" ]; then
echo "second argument not set - is used as server port!"
exit 1
fi

if [ "$PDS_SERVER_VERSION" = "" ]; then
echo "third argument not set - is used as PDS server verion!"
exit 1
fi
if [ "$PDS_SERVER_PORT" = "" ]; then
echo "fourth argument not set - is used as PDS server port!"
exit 1
fi

SCRIPT_DIR="$(dirname -- "$0")"
if [ "$SCRIPT_DIR" = "." ]; then
pwd
SCRIPT_DIR="$(pwd)"
fi
echo "SCRIPT_DIR = $SCRIPT_DIR"
cd ${SCRIPT_DIR}

cd ../.. #github action scan folder
GHA_SCAN_FOLDER_PATH="$(pwd)"
RUNTIME_DIR="${GHA_SCAN_FOLDER_PATH}/runtime";
echo "RUNTIME_DIR = $RUNTIME_DIR"

SHARED_VOLUME="${RUNTIME_DIR}/shared-volume"
mkdir "$SHARED_VOLUME" -p

## ----------------------------------
## SecHub Server
## ----------------------------------

SERVER_FOLDER_PATH="${RUNTIME_DIR}/server/${SERVER_VERSION}"
SERVER_EXECUTABLE_NAME="sechub-server-${SERVER_VERSION}.jar"
SERVER_EXECUTABLE_PATH="${SERVER_FOLDER_PATH}/${SERVER_EXECUTABLE_NAME}"
SERVER_DOWNLOAD_URL="https://github.com/mercedes-benz/sechub/releases/download/v${SERVER_VERSION}-server/${SERVER_EXECUTABLE_NAME}"
SERVER_CERTFILE_PATH="${SERVER_FOLDER_PATH}/generated-localhost-certificate.p12"

echo "SERVER_FOLDER_PATH=$SERVER_FOLDER_PATH"

SANITY_PATH_CHECK_FOR_SCRIPT="$GHA_SCAN_FOLDER_PATH/__test__/integrationtest/01-start.sh"
if [ ! -f "$SANITY_PATH_CHECK_FOR_SCRIPT" ]; then
echo "Sanity check failed - not found: $SANITY_PATH_CHECK_FOR_SCRIPT"
exit 1
fi

## download server if not available
if [ ! -f $SERVER_EXECUTABLE_PATH ]; then
mkdir "${SERVER_FOLDER_PATH}" -p
echo "Start download from $SERVER_DOWNLOAD_URL"
curl -L ${SERVER_DOWNLOAD_URL} -o ${SERVER_EXECUTABLE_PATH}
else
echo "$SERVER_EXECUTABLE_NAME exists already - skip download"
fi

## generate SecHub test certificate
if [ ! -f $SERVER_CERTFILE_PATH ]; then
mkdir "${SERVER_FOLDER_PATH}" -p
keytool -genkey -alias tomcat -storetype PKCS12 -keyalg RSA -keysize 2048 -keystore "${SERVER_CERTFILE_PATH}" -validity 3650 -storepass 123456 --dname "CN=localhost, OU=ID"
fi


## ----------------------------------
## PDS
## ----------------------------------

# https://github.com/mercedes-benz/sechub/releases/download/v1.4.0-pds/sechub-pds-1.4.0.jar
# https://github.com/mercedes-benz/sechub/releases/download/v1.4.0-pds/sechub-pds-1.4.0.jar
PDS_SERVER_FOLDER_PATH="${RUNTIME_DIR}/pds/${PDS_SERVER_VERSION}"
PDS_SERVER_EXECUTABLE_NAME="sechub-pds-${PDS_SERVER_VERSION}.jar"
PDS_SERVER_EXECUTABLE_PATH="${PDS_SERVER_FOLDER_PATH}/${PDS_SERVER_EXECUTABLE_NAME}"
PDS_SERVER_DOWNLOAD_URL="https://github.com/mercedes-benz/sechub/releases/download/v${PDS_SERVER_VERSION}-pds/${PDS_SERVER_EXECUTABLE_NAME}"
PDS_SERVER_CERTFILE_PATH="${PDS_SERVER_FOLDER_PATH}/generated-localhost-certificate.p12"

echo "PDS_SERVER_FOLDER_PATH=$PDS_SERVER_FOLDER_PATH"

## download PDS if not available
if [ ! -f $PDS_SERVER_EXECUTABLE_PATH ]; then
mkdir "${PDS_SERVER_FOLDER_PATH}" -p
echo "Start download from $PDS_SERVER_DOWNLOAD_URL"
curl -L ${PDS_SERVER_DOWNLOAD_URL} -o ${PDS_SERVER_EXECUTABLE_PATH}
else
echo "$PDS_SERVER_EXECUTABLE_NAME exists already - skip download"
fi

## generate PDS test certificate
if [ ! -f $PDS_SERVER_CERTFILE_PATH ]; then
mkdir "${PDS_SERVER_FOLDER_PATH}" -p
keytool -genkey -alias tomcat -storetype PKCS12 -keyalg RSA -keysize 2048 -keystore "${PDS_SERVER_CERTFILE_PATH}" -validity 3650 -storepass 123456 --dname "CN=localhost, OU=ID"
fi

## create config file for pds
PDS_CONFIG_FILE="$PDS_SERVER_FOLDER_PATH/pds-config.json"
cp $SCRIPT_DIR/pds-config-template.json $PDS_CONFIG_FILE


cd $SCRIPT_DIR

# Start SecHub async
SECHUB_LOGFILE=$SERVER_FOLDER_PATH/server.log
rm $SECHUB_LOGFILE -f # remove old log files on start
echo "./start_sechub_server.sh $SERVER_PORT $SERVER_EXECUTABLE_PATH $SERVER_CERTFILE_PATH $SECHUB_LOGFILE $SHARED_VOLUME" > $SECHUB_LOGFILE
./start_sechub_server.sh $SERVER_PORT $SERVER_EXECUTABLE_PATH $SERVER_CERTFILE_PATH $SECHUB_LOGFILE $SHARED_VOLUME

# Start PDS async
PDS_LOGFILE=$PDS_SERVER_FOLDER_PATH/pds.log
rm $PDS_LOGFILE -f # remove old log files on start

echo "./start_pds.sh $PDS_SERVER_PORT $PDS_SERVER_EXECUTABLE_PATH $PDS_SERVER_CERTFILE_PATH $PDS_LOGFILE $SHARED_VOLUME $PDS_CONFIG_FILE" > $PDS_LOGFILE
./start_pds.sh $PDS_SERVER_PORT $PDS_SERVER_EXECUTABLE_PATH $PDS_SERVER_CERTFILE_PATH $PDS_LOGFILE $SHARED_VOLUME $PDS_CONFIG_FILE

./wait_sechub_server_alive.sh $SERVER_PORT
./wait_pds_alive.sh $PDS_SERVER_PORT
Loading

0 comments on commit 78e6348

Please sign in to comment.