From 73776a75a1c63a14bbe141fde385fdd8255e2529 Mon Sep 17 00:00:00 2001 From: Joe Betz Date: Wed, 11 Jan 2023 18:42:23 -0500 Subject: [PATCH] Add KEP section for warning and audit annotation support --- .../3488-cel-admission-control/README.md | 124 ++++++++++++++++-- 1 file changed, 112 insertions(+), 12 deletions(-) diff --git a/keps/sig-api-machinery/3488-cel-admission-control/README.md b/keps/sig-api-machinery/3488-cel-admission-control/README.md index 9937ffddf539..865fa22f27ea 100644 --- a/keps/sig-api-machinery/3488-cel-admission-control/README.md +++ b/keps/sig-api-machinery/3488-cel-admission-control/README.md @@ -26,6 +26,7 @@ - [Limits](#limits) - [Phase 2](#phase-2) - [Enforcement Actions](#enforcement-actions) + - [Audit Events](#audit-events) - [Namespace scoped policy binding](#namespace-scoped-policy-binding) - [CEL Expression Composition](#cel-expression-composition) - [Variables](#variables) @@ -1023,26 +1024,125 @@ this enhancement. #### Enforcement Actions -For phase 1, all violations implicitly result in a `deny` enforcement action. +For parity with admission webhooks, a validating policy may also emit audit +annotations and warnings: -For phase 2, we intend to support multiple enforcement actions. +- [Audit + annotations](https://github.com/kubernetes/kubernetes/blob/97bbf07d3f3f20332912ee411fdf75ce84425e28/staging/src/k8s.io/api/admission/v1/types.go#L142) + are key/value pairs included in the audit event for an admission request. The + audit annotation key supplied by the policy definition author will be prefixed + with the name of the `ValidatingAdmissionPolicy` and policy binding, e.g.: + `mypolicy.mygroup.example.com/mybinding.mygroup.example.com/`. +- [Warnings](https://kubernetes.io/blog/2020/09/03/warnings/#admission-webhooks) + are string messages that are returned to API clients. Warning are returned for + both requests that are accepted and requests that are rejected. -Use cases: +`ValidatingAdmissionPolicy` may declare audit annotations in the policy +definition. E.g.: -- Cluster admin would like to rollout a policies, sometimes in bulk, without +```yaml +apiVersion: admissionregistration.k8s.io/v1alpha1 +kind: ValidatingAdmissionPolicy +... +spec: + ... + validations: + - expression: + auditAnnotations: + - includeWhen: # optional field + key: "my-audit-key" + valueExpression: +``` + +Additionally, `ValidatingAdmissionPolicyBinding` resource may control how +admission is enforced. This is performed using a single field. E.g.: + +```yaml +apiVersion: admissionregistration.k8s.io/v1alpha1 +kind: ValidatingAdmissionPolicyBinding +... +spec: + enforcement: warn # optional field +``` + +- `deny`: Validation failures result in a denied request. (default beahvior if + field is unset) +- `warn`: Validation failures are reported as warnings to the client. +- `silent`: Validation failures are not reported to clients. Audit annotations + for any failed validations are still included in audit events (see + below). +- (To disable audit annotations, delete the binding) + +Systems that need to aggregate validation failures may implement a [audit +webhook +backend](https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/#webhook-backend). See +below "Audit Events" for details. + +For singleton policies, the enforcement field will be set on the policy definition. + +Metrics will include enforcement so that cluster administrators can monitor the +validation failures of a binding before setting enforcement to `deny`. + +Supported use cases: + +- A policy framework captures enforcement violations during dry run and + aggregates them. (E.g. When in DryRun mode, OPA Gatekeeper aggregates + violations and records them to the status of the constraint resource). + Including validation failures in audit events makes this possible to do + using a audit webhook backend. +- Cluster admin would like to rollout policies, sometimes in bulk, without knowing all the details of the policies. During rollout the cluster admin needs a state where the policies being rolled out cannot result in admission - rejection. + rejection. With the enforcement field on bindings, cluster admins can decide + between `silent` and `warn` as the initial state and then transition through + the states until the binding reaches `deny`, monitoring metrics and audit + events along the way. - A policy framework needs different enforcement actions at different - enforcement points. -- Cluster admin would like to set specific enforcement actions for policy - violations. + enforcement points. Since this API defines the behavior of only the admission + enforcement point, higher level constructs can map to this enforcement point + as needed. + +#### Audit Events + +All audit event keys are prefixed by +`//`. -We also intend to support multiple enforcement actions: +At Metadata audit level or higher, when a validating admission binding fails any +validation expression, details are included in the audit annotations +for the audit event under the key `validation_failures`. E.g.: -- Deny -- Audit annotation -- Client warnings +```yaml +# the audit event recorded +{ + "kind": "Event", + "apiVersion": "audit.k8s.io/v1", + "annotations": { + "mypolicy.mygroup.example.com/mybinding.mygroup.example.com/validation_failure": "{\"expression\": 1, \"message\": \"x must be greater than y\", \"enforcement\": \"deny\"}" + # other annotations + ... + } + # other fields + ... +} +``` + +Also, at Metadata audit level or higher, any audit annotations declared by the policy definition +are included with the key provided. E.g.: + +```yaml +# the audit event recorded +{ + "kind": "Event", + "apiVersion": "audit.k8s.io/v1", + "annotations": { + "mypolicy.mygroup.example.com/mybinding.mygroup.example.com/myauditkey": "my audit value" + # other annotations + ... + } + # other fields + ... +} +``` #### Namespace scoped policy binding