Skip to content

Commit

Permalink
CI Testing: End to End Tests (#345)
Browse files Browse the repository at this point in the history
* end to end nginx testing

* fix format

* fix format

* use sudo to compare against files from container mount

* sleep so stanza can parse, kill stanza when done

* try cloning log library

* use token to clone lob lib

* fix repo name

* fix expect and output paths

* handle both nginx formats

* add apache_http workflow

* dump container log

* dump container log

* 10 second sleep

* use jq with diff to prevent formatting issues

* sudo

* cannot use sudo with redirection to diff, so just format with jq before using diff

* Switch back to diff, something else is going on..

* pause and cat raw output before comparing

* sudo

* fix paths

* sort before compare

* redirect output

* sort and cat

* use jtool for comparing json files

* use jtool for comparing json files

* chmod it

* haproxy workflow

* add oracledb workflow

* single test case for oracledb

* mount plugins dir

* stop and then get stanza logs

* sleep 20 seconds instead of 10, sometimes 10 is not enough

* fix log dirg

* install jtool in its own step

* fix mount

* split oracle up. start with alert logs

* oracle audit log

* upgrade jtool and use skip timestamp for haproxy and oracle

* upgrade jtoo

* upgrade jtool

* pause, stop, logs

* listener log, oracle
  • Loading branch information
Joseph Sirianni authored Sep 29, 2021
1 parent 77b7ad1 commit 56158df
Show file tree
Hide file tree
Showing 4 changed files with 601 additions and 0 deletions.
132 changes: 132 additions & 0 deletions .github/workflows/plugin_apache_http.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
name: plugin_apache_http
on:
pull_request:
paths:
- ".github/workflows/plugin_apache_http.yml"
- "plugins/apache_http.yaml"
push:
branches:
- master
jobs:
default_format:
runs-on: ubuntu-latest
steps:
- name: Clone stanza-plugins
uses: actions/checkout@v2

- name: Clone Log Library
uses: actions/checkout@v2
with:
repository: 'observIQ/log-library'
path: log-library
token: ${{ secrets.LOG_LIBRARY_ACCESS_TOKEN }}

- name: Create Output Directory
run: mkdir output

- name: Run Stanza
run: |
docker run -d \
--name stanza \
--entrypoint /stanza_home/stanza \
-v $(pwd)/log-library/cases/apache_http/default:/input \
-v $(pwd)/output:/output \
-v $(pwd)/plugins:/plugins \
observiq/stanza:1.2.7 \
--config /input/config/config.yaml --plugin_dir /plugins
- name: Pause
run: sleep 10

- name: Stop Stanza
run: docker kill stanza

- name: Dump Stanza Logs
run: docker logs stanza

- name: Pause
run: sleep 2

- name: Sort output
run: |
sudo cat output/out > output/out.sorted
sudo cat output/out.sorted
- name: Sort expect
run: |
sudo cat log-library/cases/apache_http/default/expect/expect.json > output/expect.json.sorted
sudo cat output/expect.json.sorted
- name: Check output
run: |
wget https://github.com/jsirianni/jtool/releases/download/v0.0.4/jtool-linux-amd64
chmod +x jtool-linux-amd64
if sudo ./jtool-linux-amd64 --actual-file output/out.sorted --expect-file output/expect.json.sorted; then
echo "pass"
else
echo "fail"
exit 1
fi
observiq_format:
runs-on: ubuntu-latest
steps:
- name: Clone stanza-plugins
uses: actions/checkout@v2

- name: Clone Log Library
uses: actions/checkout@v2
with:
repository: 'observIQ/log-library'
path: log-library
token: ${{ secrets.LOG_LIBRARY_ACCESS_TOKEN }}

- name: Create Output Directory
run: mkdir output

- name: Run Stanza
run: |
docker run -d \
--name stanza \
--entrypoint /stanza_home/stanza \
-v $(pwd)/log-library/cases/apache_http/default:/input \
-v $(pwd)/output:/output \
-v $(pwd)/plugins:/plugins \
observiq/stanza:1.2.7 \
--config /input/config/config.yaml --plugin_dir /plugins
- name: Pause
run: sleep 20

- name: Stop Stanza
run: docker kill stanza

- name: Dump Stanza Logs
run: docker logs stanza

- name: Pause
run: sleep 2

- name: Sort output
run: |
sudo cat output/out > output/out.sorted
sudo cat output/out.sorted
- name: Sort expect
run: |
sudo cat log-library/cases/apache_http/observiq/expect/expect.json > output/expect.json.sorted
sudo cat output/expect.json.sorted
- name: Install Json Parser 'jtool'
run: |
wget https://github.com/jsirianni/jtool/releases/download/v0.0.4/jtool-linux-amd64
chmod +x jtool-linux-amd64
- name: Check output
run: |
if sudo ./jtool-linux-amd64 --actual-file output/out.sorted --expect-file output/expect.json.sorted; then
echo "pass"
else
echo "fail"
exit 1
fi
136 changes: 136 additions & 0 deletions .github/workflows/plugin_haproxy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
name: plugin_haproxy
on:
pull_request:
paths:
- ".github/workflows/plugin_haproxy.yml"
- "plugins/haproxy.yaml"
push:
branches:
- master
jobs:
default_format:
runs-on: ubuntu-latest
steps:
- name: Clone stanza-plugins
uses: actions/checkout@v2

- name: Clone Log Library
uses: actions/checkout@v2
with:
repository: 'observIQ/log-library'
path: log-library
token: ${{ secrets.LOG_LIBRARY_ACCESS_TOKEN }}

- name: Create Output Directory
run: mkdir output

- name: Run Stanza
run: |
docker run -d \
--name stanza \
--entrypoint /stanza_home/stanza \
-v $(pwd)/log-library/cases/haproxy/default:/input \
-v $(pwd)/output:/output \
-v $(pwd)/plugins:/stanza_home/plugins \
-v $(pwd)/plugins:/plugins \
observiq/stanza:1.2.7 \
--config /input/config/config.yaml --plugin_dir /plugins
- name: Pause
run: sleep 20

- name: Stop Stanza
run: docker kill stanza

- name: Dump Stanza Logs
run: docker logs stanza

- name: Pause
run: sleep 2

- name: Sort output
run: |
sudo cat output/out > output/out.sorted
sudo cat output/out.sorted
- name: Sort expect
run: |
sudo cat log-library/cases/haproxy/default/expect/expect.json > output/expect.json.sorted
sudo cat output/expect.json.sorted
- name: Install Json Parser 'jtool'
run: |
wget https://github.com/jsirianni/jtool/releases/download/v0.0.4/jtool-linux-amd64
chmod +x jtool-linux-amd64
- name: Check output
run: |
if sudo ./jtool-linux-amd64 --skip-timestamp --actual-file output/out.sorted --expect-file output/expect.json.sorted; then
echo "pass"
else
echo "fail"
exit 1
fi
observiq_format:
runs-on: ubuntu-latest
steps:
- name: Clone stanza-plugins
uses: actions/checkout@v2

- name: Clone Log Library
uses: actions/checkout@v2
with:
repository: 'observIQ/log-library'
path: log-library
token: ${{ secrets.LOG_LIBRARY_ACCESS_TOKEN }}

- name: Create Output Directory
run: mkdir output

- name: Run Stanza
run: |
docker run -d \
--name stanza \
--entrypoint /stanza_home/stanza \
-v $(pwd)/log-library/cases/haproxy/observiq:/input \
-v $(pwd)/output:/output \
-v $(pwd)/plugins:/plugins \
observiq/stanza:1.2.7 \
--config /input/config/config.yaml --plugin_dir /plugins
- name: Pause
run: sleep 10

- name: Stop Stanza
run: docker kill stanza

- name: Dump Stanza Logs
run: docker logs stanza

- name: Pause
run: sleep 2

- name: Sort output
run: |
sudo cat output/out > output/out.sorted
sudo cat output/out.sorted
- name: Sort expect
run: |
sudo cat log-library/cases/haproxy/observiq/expect/expect.json > output/expect.json.sorted
sudo cat output/expect.json.sorted
- name: Install Json Parser 'jtool'
run: |
wget https://github.com/jsirianni/jtool/releases/download/v0.0.4/jtool-linux-amd64
chmod +x jtool-linux-amd64
- name: Check output
run: |
if sudo ./jtool-linux-amd64 --skip-timestamp --actual-file output/out.sorted --expect-file output/expect.json.sorted; then
echo "pass"
else
echo "fail"
exit 1
fi
Loading

0 comments on commit 56158df

Please sign in to comment.