diff --git a/README.md b/README.md index d2cc9e98..1069041c 100644 --- a/README.md +++ b/README.md @@ -1,30 +1,224 @@ # json-kyverno -## create a terraform plan in json format +This CLI tool is very similar to the [Kyverno CLI](https://github.com/kyverno/kyverno/tree/main/cmd/cli/kubectl-kyverno) tool. -```console -# init terraform -terraform init +The difference is that this CLI tool can apply policies to abitrary json or yaml payloads. + +Policy definition syntax is looks a lot like the [Kyverno policy](https://kyverno.io/docs/kyverno-policies/) definition syntax but is more generic and flexible. +This was needed to allow working with arbitrary payloads, not just [Kubernetes](https://kubernetes.io) ones. +Those differences are detailed in the [section below](#differences-with-with-kyverno-policy-definition-syntax). + +Additionally, you can provide preprocessing queries in [jmespath](https://jmespath.site) format to preprocess the input payload before evaluating *resources* against policies. +This is necessary if the input payload is not what you want to directly analyse. +Preprocessing is detailed in the following [section](#preprocessing). + +## Differences with with Kyverno policy definition syntax + +Sections below highlight the main differences between polcies used by this tool and [Kyverno policies](https://kyverno.io/docs/kyverno-policies/). + +### Different `apiVersion` and `kind` + +Both [Kyverno policies](https://kyverno.io/docs/kyverno-policies/) and policies used by this tool are defined using [Kubernetes](https://kubernetes.io) manifests. + +They don't use the same `apiVersion` and `kind` though. + +[Kyverno policies](https://kyverno.io/docs/kyverno-policies/) belong to the `kyverno.io` group, exist in multiple versions (`v1`, `v2beta1`) and can be of kind `Policy` or `ClusterPolicy`. + +Policies for this tool belong to the `json.kyverno.io` group, exist only in `v1alpha1` version and can only be of kind `Policy`. + +```yaml +apiVersion: json.kyverno.io/v1alpha1 +kind: Policy +metadata: + name: test +spec: + rules: + - name: foo-bar-4 + validate: + pattern: + foo: + bar: 4 +``` + +The concept of clustered vs namespaced resources exist only in the [Kubernetes](https://kubernetes.io) world and it didn't make sense to reproduce the same pattern in this tool. + +### Different `match` and `exclude` statements + +Both [Kyverno policies](https://kyverno.io/docs/kyverno-policies/) and policies used by this tool can match and exclude *resources* when being evaluated. + +[Kyverno policies](https://kyverno.io/docs/kyverno-policies/) use [Kubernetes](https://kubernetes.io) specific constructs for that matter that didn't map well with arbitrary payloads. -# create a plan -terraform plan -out=tf.plan +This tool uses partial resource definitions to implement `match` and `exclude` statements (the approach was inspired by [kuttl](https://github.com/kudobuilder/kuttl)): -# show plan in json -terraform show -json tf.plan > tf.plan.json +```yaml +apiVersion: json.kyverno.io/v1alpha1 +kind: Policy +metadata: + name: required-s3-tags +spec: + rules: + - name: require-team-tag + match: + any: + - resource: + type: aws_s3_bucket + exclude: + any: + - resource: + name: bypass-* + validate: + pattern: + values: + tags: + Team: ?* ``` -## build json-kyverno +In the example above, every *resource* having `type: aws_s3_bucket` will match, and *resources* having `name: bypass-*` will be excluded. + +Note that wildcards are supported when evaluating string fields. + +### Different jmesPath implementation + +This tool uses [jmespath-community/go-jmespath](https://github.com/jmespath-community/go-jmespath), a more modern implementation than the one used in [Kyverno](https://kyverno.io). + +This implementation supports the `let` feature and this tool leverages it to implement context entries: + +```yaml +apiVersion: json.kyverno.io/v1alpha1 +kind: Policy +metadata: + name: required-s3-tags +spec: + rules: + - name: require-team-tag + match: + any: + - resource: + type: aws_s3_bucket + context: + - name: expectedTeam + variable: + value: Kyverno + validate: + message: Bucket `{{ resource.name }}` ({{ resource.address }}) does not have the required Team tag {{ $expectedTeam }} + pattern: + values: + tags: + Team: '{{ $expectedTeam }}' +``` + +### No preconditions, pattern operators or anchors + +Policies used by this tool don't support `preconditions`, pattern operators or anchors. + +Most of the time `preconditions` can be replaced by the more flexible `match` and `exclude` statements. + +Pattern operators and anchors can be replaced with an enhanced pattern syntax detailed [below](#enhanced-pattern-syntax). + +### Enhanced pattern syntax + +[Kyverno policies](https://kyverno.io/docs/kyverno-policies/) started with a declarative approach but slowly adopted the imperative approach too, because of the limitations in the implemented declarative approach. + +This tool tries to be as declarative as possible, for now `forEach`, pattern operators and anchors are not supported are not supported. +Hopefully we won't need to adopt an imperative approach anymore. + +Instead, the pattern syntax can now be used to express complex and dynamic conditions by using [jmespath](https://jmespath.site) expressions as pattern keys. + +Given the input payload below: + +```yaml +foo: + baz: true + bar: 4 + bat: 6 +``` + +It is now possible to write a validation pattern like this: + +```yaml +apiVersion: json.kyverno.io/v1alpha1 +kind: Policy +metadata: + name: test +spec: + rules: + - name: foo-bar-4 + validate: + pattern: + foo: + '{{ bar > `3` }}': true + '{{ !baz }}': false + '{{ bar + bat }}': 10 +``` + +As the expected pattern is traversed, the current node in the actual *resource* is made available to eventual [jmespath](https://jmespath.site) expressions defined in child keys. + +When a key is a [jmespath](https://jmespath.site) expression, the reult of the evaluation will be matched against the value in the pattern. + +Of course, the expression can return a complex object or array. This object (or array) will then be traversed and compared against the value as if it was the actual *resource*. + +## SDK + +This CLI tool contains an initial implementation of an SDK to allow flexible creation of dedicated policy engines. + +The [json-engine](./pkg/json-engine/) at the heart of this tool is built by assembling blocks provided by the [engine](./pkg/engine/) SDK. + + +## Build json-kyverno + +To build this tool locally, simply run: ```console make build ``` -## invoke json-kyverno +## Preprocessing -```console -# with json payload -./json-kyverno --payload ./testdata/tf-plan/tf.plan.json --pre-process "planned_values.root_module.resources" --policy ./testdata/tf-plan/policy.yaml +When the input payload is not what you want to analyse directly, you can provide one or more [jmespath](https://jmespath.site) expressions to preprocess the data. +The policies will be evaluated against the result of the preprocessing step. + +Traditionnally, policies apply to *resources* and this is how this tool implements policy evaluation: + +``` +loop through all resources { + loop through all policies { + loop through all rules in the policy { + evaluate the resource against the rule + } + } +} +``` + +Note that if you provide a single payload, the tool will internally wrap it in an array of one element. +So imagine an input payload similar to this: + +```yaml +version: 1.2.3 +creationDate: '2023-09-29' +resources: +- type: something + name: foo + spec: + # ... +- type: something else + name: bar + spec: + # ... +``` + +The *resources* you want to analyse are located under the `resources` stanza, and your policies are probably written to work on those *resources*. +In order to extract the data under the `resources` stanza before processing happens you can specify the `--pre-process "resources"` when invoking the tool. + +You can chain mutliple preprocessing queries by specifying the `--pre-process` flag multiple times. +There is no limitation in a preprocessing [jmespath](https://jmespath.site) expression. + +## Invoke json-kyverno + +```console # with yaml payload -./json-kyverno --payload ./testdata/payload-yaml/payload.yaml --pre-process "planned_values.root_module.resources" --policy ./testdata/payload-yaml/policy.yaml +./json-kyverno --payload ./testdata/foo-bar/payload.yaml --policy ./testdata/foo-bar/policy.yaml + +# with json payload (and pre processing) +./json-kyverno --payload ./testdata/tf-plan/tf.plan.json --pre-process "planned_values.root_module.resources" --policy ./testdata/tf-plan/policy.yaml ```