Skip to content

Commit

Permalink
Merge pull request #366 from jhkrug/pol-testing
Browse files Browse the repository at this point in the history
Refresh tutorial on policy testing.
  • Loading branch information
flavio authored Feb 28, 2024
2 parents a73136c + 6c173f9 commit 56d5100
Show file tree
Hide file tree
Showing 3 changed files with 79 additions and 56 deletions.
42 changes: 25 additions & 17 deletions docs/tutorials/testing-policies/02-policy-authors.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,49 +15,56 @@ doc-topic: [testing-policies, policy-authors]
Kubewarden policies are regular programs compiled as WebAssembly (Wasm).
As with any kind of program, good test coverage is important.

Policy authors can use their favorite development environments. You can use familiar tools, and testing frameworks to verify development.
Policy authors can use their favorite development environments.
You can use familiar tools, and testing frameworks to verify development.

These two Kubewarden policies provide an example written in
These Kubewarden policies are examples written in
[Rust](../writing-policies/rust/01-intro-rust.md) and
[Go](../writing-policies/go/01-intro-go.md):

- [psp-apparmor](https://github.com/kubewarden/psp-apparmor)
- [ingress-policy](https://github.com/kubewarden/ingress-policy)

They both have test suites using standard testing for their development environments.
They have test suites using standard testing for their development environments.

The policies use GitHub Actions for their CI pipelines.

## End-to-end tests

You can also write tests that execute against the Wasm binary containing your policy.
This can be done without having to deploy a Kubernetes cluster by using these tools:
To do this without having to deploy a Kubernetes cluster you can use these tools:

- [bats](https://github.com/bats-core/bats-core): is used to write tests and automate their execution.
- [bats](https://github.com/bats-core/bats-core): is to write tests and automate their execution.
- [kwctl](https://github.com/kubewarden/kwctl): Kubewarden's default CLI tool that helps you with policy-related operations; pull, inspect, annotate, push, and run.

To use `kwctl run` the following input is needed:
To use `kwctl run` you need the following:

1. Wasm binary file reference of the policy to be run.
The Kubewarden policy can be loaded from the local filesystem (`file://`), an HTTP(s) server (`https://`), or an OCI registry (`registry://`).
1. The admission request object to be tested.
You provide it via the `--request-path` argument.
Or you can provide it on `stdin` by setting `--request-path` to `-`.
1. Provide policy settings for run time as an inline JSON via `--settings-json` flag.
Or a JSON, or a YAML file loaded from the filesystem via `--settings-path`.
1. The Wasm binary file reference of the policy to run.
The Kubewarden policy can be loaded from:
- the local filesystem (`file://`)
- a HTTP(s) server (`https://`
- an OCI registry (`registry://`).
1. The admission request object to test.
You give it via the `--request-path` argument,
or on `stdin` by setting `--request-path` to `-`.
1. The policy settings for runtime as an inline JSON via `--settings-json` flag.
Or a JSON, or a YAML file, loaded from the file system via `--settings-path`.

After the test `kwctl` prints the `ValidationResponse` object to standard output.
After the test `kwctl`, prints the `ValidationResponse` object to standard output.

This is how you use `kwctl` to test the Wasm binary of `ingress-policy` linked to above:
This is how you use `kwctl` to test the Wasm binary of the [`ingress-policy`](https://github.com/kubewarden/ingress-policy) mentioned previously:

```
```console
$ curl https://raw.githubusercontent.com/kubewarden/ingress-policy/v0.1.8/test_data/ingress-wildcard.json 2> /dev/null | \
kwctl run \
--settings-json '{"allowPorts": [80], "denyPorts": [3000]}' \
--request-path - \
registry://ghcr.io/kubewarden/policies/ingress:v0.1.8 | jq
```

You can download pre-built binaries of `kwctl`
[here](https://github.com/kubewarden/kwctl/releases).

Using `bats` you can write a test that runs this command and looks for the expected outputs:

<details>
Expand Down Expand Up @@ -93,4 +100,5 @@ $ bats e2e.bats
1 tests, 0 failures
```

[This](../writing-policies/go/05-e2e-tests.md) section of the documentation has more about writing end-to-end tests of your policies.
[This](../writing-policies/go/05-e2e-tests.md)
section of the documentation has more about writing end-to-end tests of your policies.
81 changes: 48 additions & 33 deletions docs/tutorials/testing-policies/03-cluster-operators.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,39 +12,47 @@ doc-topic: [testing-policies, cluster-operators]
<link rel="canonical" href="https://docs.kubewarden.io/tutorials/testing-policies/cluster-operators"/>
</head>

As a Kubernetes cluster operator, you will want to perform testing against Kubewarden policies you want to use.
As a Kubernetes cluster operator, you'll want to perform testing for Kubewarden policies you want to use.

You will have questions like:
You'll have questions like:

- What are the correct policy settings to get the validation/mutation outcome I need?
- How can I be sure everything will keep working as expected when:
- I upgrade the policy to a newer version
- I add/change some Kubernetes resources
- I change the configuration parameters of the policy
- and so on?
- What are the correct policy settings to get the validation/mutation outcome needed?
- How can I be sure everything keeps working as expected when I:
- upgrade the policy to a newer version?
- add/change Kubernetes resources?
- change the configuration parameters of the policy?
- and so forth?

Kubewarden has a utility, [`kwctl`](https://github.com/kubewarden/kwctl), that allows testing of the policies outside of Kubernetes.
Kubewarden has a utility, [`kwctl`](https://github.com/kubewarden/kwctl),
that permits testing of the policies outside of Kubernetes.

To use `kwctl` we invoke it with following inputs:
To use `kwctl` you invoke it with following inputs:

1. A WebAssembly binary file URI of the policy to be run.
The Kubewarden policy can be loaded from the local filesystem `file://`, an HTTP(s) server `https://`, or an OCI registry `registry://`.
1. The admission request object to be evaluated.
You provide it with the `--request-path` argument.
The Kubewarden policy can load from the:
- local filesystem `file://`
- a HTTP(s) server `https://`
- an OCI registry `registry://`.
1. The admission request object to test.
You give it with the `--request-path` argument.
Use `stdin` by setting `--request-path` to `-`.
1. You provide run time policy settings as inline JSON via `--settings-json` flag. Or with a JSON or YAML file from the filesystem using `--settings-path`.
1. The policy settings for runtime as an inline JSON via `--settings-json` flag.
Or a JSON, or a YAML file, loaded from the file system via `--settings-path`.

After the test `kwctl` prints the `ValidationResponse` object to the standard output.
After the test `kwctl`, prints the `ValidationResponse` object to the standard output.

You can download pre-built binaries of `kwctl` from [here](https://github.com/kubewarden/kwctl/releases).
You can download pre-built binaries of `kwctl`
[here](https://github.com/kubewarden/kwctl/releases).

## A testing example

This section describes how to test the [psp-apparmor](https://github.com/kubewarden/psp-apparmor) policy with different configurations and validation request objects.
This section describes how to test the
[psp-apparmor](https://github.com/kubewarden/psp-apparmor)
policy with different configurations and validation request objects.

### Create `AdmissionReview` requests

We have to create files holding the `AdmissionReview` objects to test the policy.
You need to create files holding the `AdmissionReview` objects to test the policy.

You can create a file named `pod-req-no-specific-apparmor-profile.json` with the following contents:

Expand Down Expand Up @@ -84,8 +92,9 @@ You can create a file named `pod-req-no-specific-apparmor-profile.json` with the

</details>

This request tries to create a Pod that doesn't specify any AppArmor profile to be used.
Because it doesn't have an `annotation` with the `container.apparmor.security.beta.kubernetes.io/<container-name>` key.
This request tries to create a Pod that doesn't specify any AppArmor profile to use.
This is because it doesn't have an `annotation` with the
`container.apparmor.security.beta.kubernetes.io/<container-name>` key.

You can create a file named `pod-req-apparmor-unconfined.json` with the
following contents:
Expand Down Expand Up @@ -130,9 +139,11 @@ following contents:
</details>

This request tries to create a Pod with a container called `nginx` running with the `unconfined` AppArmor profile.
Note that, running in `unconfined` mode is a bad security practice.
This is for tutorial purposes only.
Running in `unconfined` mode is a bad security practice.

Now you can create a file named `pod-req-apparmor-custom.json` with the following contents:
Now you can create a file named
`pod-req-apparmor-custom.json` with the following contents:

<details>

Expand Down Expand Up @@ -175,12 +186,12 @@ Now you can create a file named `pod-req-apparmor-custom.json` with the followin

:::note
These are all simplified `AdmissionReview` objects.
We have only the fields relevant to our testing of the policy.
Only the fields relevant to our testing of the policy are used.
:::

### Test the policy

Now we can use `kwctl` to test the creation of a Pod not specifying an AppArmor profile:
Now you can use `kwctl` to test the creation of a Pod not specifying an AppArmor profile:

```console
$ kwctl run \
Expand All @@ -189,7 +200,7 @@ $ kwctl run \
| jq
```

The policy will accept the request and produce output like:
The policy accepts the request and produces output like:

```console
{
Expand All @@ -198,8 +209,7 @@ The policy will accept the request and produce output like:
}
```

The policy will instead reject the creation of a Pod with an
`unconfined` AppArmor profile:
The policy rejects the creation of a Pod with an `unconfined` AppArmor profile:

```console
$ kwctl run \
Expand All @@ -215,7 +225,10 @@ $ kwctl run \
}
```

Both times we ran the policy **without** providing any kind of setting. As the [policy's documentation](https://github.com/kubewarden/psp-apparmor#configuration) states, this results in preventing the usage of non-default profiles.
On both occasions you ran the policy **without** providing any kind of setting.
As the
[policy's documentation](https://github.com/kubewarden/psp-apparmor#configuration)
states, this results in preventing the usage of non-default profiles.

The Pod using a custom `nginx` profile gets rejected by the policy too:

Expand All @@ -233,7 +246,7 @@ $ kwctl run \
}
```

You can change the default behavior, allowing some chosen AppArmor to be used:
You can change the default behavior, allowing chosen AppArmor profiles to be used:

```console
$ kwctl run \
Expand All @@ -254,11 +267,12 @@ Now the request succeeds:

## Automation

All these steps, shown above, can be automated using [bats](https://github.com/bats-core/bats-core).
You can automate all these steps using
[bats](https://github.com/bats-core/bats-core).

You can write a series of tests and integrate their execution inside your existing CI and CD pipelines.

The commands above can be "wrapped" into a `bats` test:
The commands can be "wrapped" into a `bats` test:

<details>

Expand Down Expand Up @@ -292,7 +306,7 @@ The commands above can be "wrapped" into a `bats` test:

</details>

If the `bats` code above is in the file `e2e.bats`, we can run the test as:
If the `bats` code is in the file `e2e.bats`, you can run the test as:

```console
$ bats e2e.bats
Expand All @@ -302,4 +316,5 @@ $ bats e2e.bats
2 tests, 0 failures
```

[This](../writing-policies/go/05-e2e-tests.md) section has more about writing end-to-end tests for your policies.
[This](../writing-policies/go/05-e2e-tests.md)
section has more about writing end-to-end tests for your policies.
12 changes: 6 additions & 6 deletions docs/tutorials/testing-policies/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
sidebar_label: Policy testing
title: Policy testing
description: The two personas using Kubewarden policies are policy authors and cluster operators.
description: A tutorial about testing policies in Kubewarden.
keywords: [kubewarden, persons, policy author, cluster operator]
doc-persona: [kubewarden-operator, kubewarden-policy-developer]
doc-type: [tutorial]
Expand All @@ -15,11 +15,11 @@ doc-topic: [testing-policies, introduction]
## Policy testing personas

This section covers the topic of testing Kubewarden Policies.
We define two personas interested in testing policies:
There are two personas interested in testing policies:

- **policy author**: you're writing a Kubewarden Policy and you want to ensure your code behaves the way you expect.
- **Policy developer**: you're writing a Kubewarden Policy and you want to make sure your code behaves the way you expect.

- **cluster operator**: you found a Kubewarden Policy and you want to test and tune the policy settings before deploying it.
Maybe you want to keep testing these settings inside of your CI/CD pipelines,...
- **Cluster operator**: you found a Kubewarden Policy and you want to test and tune the policy settings before deploying it.
You want to keep testing these settings in your CI/CD pipelines, and other administrative tasks.

The next sections of the documentation shows how Kubewarden policies might be tested by these personas.
The next sections of the documentation shows how these personas can test Kubewarden policies.

0 comments on commit 56d5100

Please sign in to comment.