Update documents to cover CEL validation on CRDs (nginx#1212)

Update resource validation docs to cover CEL validation. Problem: Gateway API 0.8.0 introduced CEL based-validation and the shift away from webhook validation. Solution: Updated the resource validation docs to cover CEL validation for supported Kubernetes versions and kept webhook validation for older versions.
miledxz · Nov 8, 2023 · a62ed55 · a62ed55
1 parent a012512
commit a62ed55
Show file tree

Hide file tree

Showing 2 changed files with 63 additions and 14 deletions.
diff --git a/design/resource-validation.md b/design/resource-validation.md
@@ -30,9 +30,20 @@ To help the implementations with the validation, the Gateway API already include
   X must be a string) and the contents of the fields (for example, field Y only allows values 'a' and 'b').
   Additionally, it enforces the limits like max lengths on field values. Note:
   Kubernetes API server enforces this validation. To bypass it, a user needs to change the CRDs.
-- *The webhook validation*. This validation is written in go and run as part of the webhook, which is included in the
-  Gateway API installation files. The validation covers additional logic, not possible to implement in the CRDs. It does
-  not repeat the validation from the CRDs. Note: a user can bypass this validation if the webhook is not installed.
+
+#### For Kubernetes 1.25+
+
+- *CEL Validation*. This validation is embedded in the Gateway API CRDs and covers additional logic not available in the
+  OpenAPI schema validation. For example, the field X must be specified when type is set to Y; or X must be nil if
+  Y is not Z. Note: Kubernetes API server enforces this validation. To bypass it, a user needs to change the CRDs.
+
+#### For Kubernetes 1.23 and 1.24
+
+- *The webhook validation*. This validation is written in go and ran as part of the webhook, which is included in the
+  Gateway API installation files. The validation covers additional logic, not possible to implement in the OpenAPI
+  schema validation.
+  It does not repeat the OpenAPI schema validation from the CRDs. Note: a user can bypass this validation if the webhook
+  is not installed.
 
 However, the built-in validation rules do not cover all validation needs of NGF:
 
@@ -109,7 +120,7 @@ If a resource is invalid:
 
 ### NGF-specific validation
 
-After re-running the webhook validation, NGF will run NGF-specific validation, written in go.
+After re-running the webhook validation, NGF will run NGF-specific validation written in go.
 
 NGF-specific validation will:
 
@@ -127,16 +138,28 @@ If a resource is invalid, NGF will report the error in its status.
 
 ### Summary of Validation
 
-The table below summarizes the validation methods NGF will use. Any Gateway API resource will be validated by the
+The tables below summarize the validation methods NGF will use. Any Gateway API resource will be validated by the
 following methods in order of their appearance in the table.
 
+#### For Kubernetes 1.25+
+
+| Name                         | Type                       | Component             | Scope                   | Feedback loop for errors                                                         | Can be bypassed?               |
+|------------------------------|----------------------------|-----------------------|-------------------------|----------------------------------------------------------------------------------|--------------------------------|
+| CRD validation               | OpenAPI and CEL validation | Kubernetes API server | Structure, field values | Kubernetes API server returns any errors a response for an API call.             | Yes, if the CRDs are modified. |
+| Re-run of webhook validation | Go code                    | NGF control plane     | Field values            | Errors are reported as Event for the resource.                                   | No                             |
+| NGF-specific validation      | Go code                    | NGF control plane     | Field values            | Errors are reported in the status of a resource after its creation/modification. | No                             |
+
+
+#### For Kubernetes 1.23 and 1.24
+
 | Name                         | Type    | Component             | Scope                   | Feedback loop for errors                                                         | Can be bypassed?                                                                     |
 |------------------------------|---------|-----------------------|-------------------------|----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|
 | CRD validation               | OpenAPI | Kubernetes API server | Structure, field values | Kubernetes API server returns any errors a response for an API call.             | Yes, if the CRDs are modified.                                                       |
 | Webhook validation           | Go code | Gateway API webhook   | Field values            | Kubernetes API server returns any errors a response for an API call.             | Yes, if the webhook is not installed, misconfigured, or running a different version. |
 | Re-run of webhook validation | Go code | NGF control plane     | Field values            | Errors are reported as Event for the resource.                                   | No                                                                                   |
 | NGF-specific validation      | Go code | NGF control plane     | Field values            | Errors are reported in the status of a resource after its creation/modification. | No                                                                                   |
 
+
 Notes:
 
 - The amount and the extent of the validation should allow multiple application developers to share a single NGF (User
@@ -148,7 +171,7 @@ Notes:
 
 NGF will support more resources:
 
-- More Gateway API resources. For those, NGF will use the four validation methods from the table in the previous
+- More Gateway API resources. For those, NGF will use the validation methods from the table in the previous
   section.
 - Introduce NGF resources. For those, NGF will use CRD validation (the rules of which are fully controlled by us). The
   CRD validation will include the validation to prevent invalid NGINX configuration values and malicious values. Because

diff --git a/docs/resource-validation.md b/docs/resource-validation.md
@@ -15,6 +15,15 @@ created.
 
 A Gateway API resource (a new resource or an update for the existing one) is validated by the following steps:
 
+### For Kubernetes 1.25+
+
+1. OpenAPI schema validation by the Kubernetes API server.
+2. CEL validation by the Kubernetes API server.
+3. Webhook validation by NGF.
+4. Validation by NGF.
+
+### For Kubernetes 1.23 and 1.24
+
 1. OpenAPI schema validation by the Kubernetes API server.
 2. Webhook validation by the Gateway API webhook.
 3. Webhook validation by NGF.
@@ -64,35 +73,52 @@ The HTTPRoute "coffee" is invalid: spec.hostnames[0]: Invalid value: "cafe.!@#$%
 > While unlikely, bypassing this validation step is possible if the Gateway API CRDs are modified to remove the validation.
 > If this happens, Step 4 will reject any invalid values (from NGINX perspective).
 
-### Step 2 - Webhook Validation by Gateway API Webhook
+### Step 2 - For Kubernetes 1.25+ - CEL Validation by Kubernetes API Server
+
+The Kubernetes API server validates Gateway API resources using CEL validation embedded in the Gateway API CRDs.
+It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation.
+For example, if you create a Gateway resource with a TCP listener that configures a hostname, the CEL validation will
+reject it with the following error:
+
+
+```shell
+kubectl apply -f some-gateway.yaml
+```
+
+```text
+The Gateway "some-gateway" is invalid: spec.listeners: Invalid value: "array": hostname must not be specified for protocols ['TCP', 'UDP']
+```
+
+More information on CEL in Kubernetes can be found [here](https://kubernetes.io/docs/reference/using-api/cel/).
+
+### Step 2 - For Kubernetes 1.23 and 1.24 - Webhook Validation by Gateway API Webhook
 
 The Gateway API comes with a validating webhook which is enabled by default in the Gateway API installation manifests.
 It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation. For example, if
 you create a Gateway resource with a TCP listener that configures a hostname, the webhook will reject it with the
 following error:
 
 ```shell
-kubectl apply -f prod-gateway.yaml
+kubectl apply -f some-gateway.yaml
 ```
 
 ```text
-Error from server: error when creating "prod-gateway.yaml": admission webhook "validate.gateway.networking.k8s.io" denied the request: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP
+Error from server: error when creating "some-gateway.yaml": admission webhook "validate.gateway.networking.k8s.io" denied the request: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP
 ```
 
 > Bypassing this validation step is possible if the webhook is not running in the cluster.
 > If this happens, Step 3 will reject the invalid values.
 
 ### Step 3 - Webhook validation by NGF
-
-The previous step relies on the Gateway API webhook running in the cluster. To ensure that the resources are validated
-with the webhook validation rules, even if the webhook is not running, NGF performs the same validation. However, NGF
-performs the validation *after* the Kubernetes API server accepts the resource.
+To ensure that the resources are validated with the webhook validation rules, even if the webhook is not running,
+NGF performs the same validation. However, NGF performs the validation *after* the Kubernetes API server accepts
+the resource.
 
 Below is an example of how NGF rejects an invalid resource (a Gateway resource with a TCP listener that configures a
 hostname) with a Kubernetes event:
 
 ```shell
-kubectl describe gateway prod-gateway
+kubectl describe gateway some-gateway
 ```
 
 ```text