Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Update README will all new features #65

Merged
merged 2 commits into from
Sep 4, 2022
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 54 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,24 @@ go install github.com/fzipi/go-ftw@latest
To run tests you need:
1. a WAF (doh!)
2. a file where the waf stores the logs
3. a config file, or environment variables, with the information to get the logs and how to parse them (I might embed this for the most commonly used, like Apache/NGiNX)
3. a config file, or environment variables, with the information to get the logs and how to parse them (I might embed this for the most commonly used, like Apache/NGiNX), and additional options described next.

By default, _ftw_ would search for a file in `$PWD` with the name `.ftw.yaml`. Example configurations for `apache` and `nginx` below:
### YAML Config file

With a configuration file you can alter the test results or just provide neeed paths for files and enabling features. The config file has four basic values:
fzipi marked this conversation as resolved.
Show resolved Hide resolved

```yaml
logfile: <the relative patch to the waf logfile>
fzipi marked this conversation as resolved.
Show resolved Hide resolved
logmarkerheadername: <a header name used for log parsing (see "How log parsing works" below)>
testoverride: <a list of things to override (see "Overriding tests" below)>
mode: "default" or "cloud" (only change it if you need "cloud")
```

By default, _ftw_ would search for a file in `$PWD` with the name `.ftw.yaml`. You can pass the `--config <config file name>` to point it to a different file.

### Logfile

Running in default mode implies you have access to a logfile to contrast the WAF behavior with test results. Example configurations for `apache` and `nginx` can be found below:
fzipi marked this conversation as resolved.
Show resolved Hide resolved

```yaml
---
Expand All @@ -51,6 +66,8 @@ logfile: '../coreruleset/tests/logs/modsec2-apache/apache2/error.log'
logfile: '../coreruleset/tests/logs/modsec3-nginx/nginx/error.log'
```

### WAF Server

I normally perform my testing using the [Core Rule Set](https://github.com/coreruleset/coreruleset/).

You can start the containers from that repo using docker-compose:
fzipi marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -60,6 +77,8 @@ git clone https://github.com/coreruleset/coreruleset.git
docker-compose -f tests/docker-compose.yml up -d modsec2-apache
fzipi marked this conversation as resolved.
Show resolved Hide resolved
```

## Running

This is the help for the `run` command:
```bash
❯ ftw run -h
Expand All @@ -79,10 +98,10 @@ Flags:
-t, --time show time spent per test

Global Flags:
--config string override config file (default is $PWD/.ftw.yaml) (default "c")
--cloud cloud mode: rely only in http status code for determining test succes or failure (assumes no logs access)
fzipi marked this conversation as resolved.
Show resolved Hide resolved
--config string override config file (default is $PWD/.ftw.yaml)
--debug debug output
--trace trace output: really, really verbose

```

Here's an example on how to run your tests:
Expand Down Expand Up @@ -122,9 +141,12 @@ Happy testing!

## Additional features

You can add functions to your tests, to simplify bulk writing, or even read values from the environment while executing. This is because `data:` sections in tests are parsed with Go [text/template](https://golang.org/pkg/text/template/), and also are given the power of additional [Sprig functions](https://masterminds.github.io/sprig/).
- templates with the power of Go [text/template](https://golang.org/pkg/text/template/). Add your template to any `data:` sections and enjoy!
- [Sprig functions](https://masterminds.github.io/sprig/) can be added to templates as well.
- Override test results.
- Cloud mode! The new mode will override test results and only base the test result on http status code answers.
fzipi marked this conversation as resolved.
Show resolved Hide resolved

This will allow you to write tests like this:
With templates and functions you can simplify bulk test writing, or even read values from the environment while executing. This features allow you to write tests like this:

```yaml
data: 'foo=%3d{{ "+" | repeat 34 }}'
Expand All @@ -150,15 +172,25 @@ data: 'username=fzipi

Other interesting functions you can use are: `randBytes`, `htpasswd`, `encryptAES`, etc.

## Overriding test results
## Overriding tests

Sometimes you have tests that work well for some platform combinations, e.g. Apache + modsecurity2, but fail for others, e.g. NGiNX + modsecurity3. Taking that into account, you can override test results using the `testoverride` config param. The test will be run, but the _result_ would be overriden, and your comment will be printed out.

Example:
Tests can be altered using four lists:
- `input` allows you to override global parameters in tests. An example usage is if you want to change the `dest_addr` of all tests to point to an external IP or host.
fzipi marked this conversation as resolved.
Show resolved Hide resolved
- `ignore` is for tests you want to ignore. It will still execute the test, but ignore the results. You should add a comment on why you ignore the test
fzipi marked this conversation as resolved.
Show resolved Hide resolved
- `forcepass` is for tests you want to pass unconditionally. Test will be executed, and pass even when the test fails. You should add a comment on why you force pass the test
- `forcefail` is for tests you want to fail unconditionally. Test will be executed, and fail even when the test passes. You should add a comment on why you force fail the test

Example using all the lists above:

```yaml
...
testoverride:
input:
dest_addr: "192.168.1.100"
port: 8080
protocol: "http"
ignore:
# text comes from our friends at https://github.com/digitalwave/ftwrunner
'941190-3': 'known MSC bug - PR #2023 (Cookie without value)'
Expand All @@ -173,6 +205,20 @@ testoverride:

You can combine any of `ignore`, `forcefail` and `forcepass` to make it work for you.

## Cloud mode

Most of the tests rely on having access to a logfile to check for success or failure. Sometimes that is not possible, for example, when testing cloud services or servers where you don't have access to logfiles and/or logfiles won't have the information you need to decide if the test was good or bad.

With cloud mode we move the decision on test failure or success to the HTTP status code received after performing the test. The general idea is that you setup your WAF in blocking mode, so anything matching will return a block status (e.g. 403), and if not we expect a 2XX return code.
fzipi marked this conversation as resolved.
Show resolved Hide resolved

An example config file for this is:
```
---
mode: 'cloud'
```

Or you can just run: `./ftw run --cloud`

## How log parsing works
The log output from your WAF is parsed and compared to the expected output.
The problem with log files is that they aren't updated in real time, e.g. because the
Expand Down