Skip to content

Commit

Permalink
Merge branch 'main' into status-order
Browse files Browse the repository at this point in the history
  • Loading branch information
pleshakov authored Dec 14, 2023
2 parents 9e9ecc6 + 7e075c6 commit 50be6f4
Show file tree
Hide file tree
Showing 52 changed files with 46,976 additions and 131 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ jobs:
if: github.ref_type == 'tag'

- name: Install Cosign
uses: sigstore/cosign-installer@1fc5bd396d372bee37d608f955b336615edf79c8 # v3.2.0
uses: sigstore/cosign-installer@9614fae9e5c5eddabb09f90a270fcb487c9f7149 # v3.3.0
if: github.ref_type == 'tag'

- name: Build binary
Expand Down Expand Up @@ -313,7 +313,7 @@ jobs:
NGINX_CONF_DIR=internal/mode/static/nginx/conf
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@22d2755f774d925b191a185b74e782a4b0638a41 # 0.15.0
uses: aquasecurity/trivy-action@91713af97dc80187565512baba96e4364e983601 # 0.16.0
continue-on-error: true
with:
image-ref: ghcr.io/nginxinc/nginx-gateway-fabric${{ matrix.container == 'nginx' && '/nginx' || '' }}:${{ steps.meta.outputs.version }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/update-docker-images.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ jobs:
NGINX_CONF_DIR=internal/mode/static/nginx/conf
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@22d2755f774d925b191a185b74e782a4b0638a41 # 0.15.0
uses: aquasecurity/trivy-action@91713af97dc80187565512baba96e4364e983601 # 0.16.0
continue-on-error: true
with:
image-ref: ghcr.io/nginxinc/nginx-gateway-fabric/nginx:${{ needs.variables.outputs.ngf_tag }}
Expand Down
2 changes: 1 addition & 1 deletion build/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ RUN go mod download
COPY . /go/src/github.com/nginxinc/nginx-gateway-fabric
RUN make build

FROM alpine:3.18 as capabilizer
FROM alpine:3.19 as capabilizer
RUN apk add --no-cache libcap

FROM capabilizer as local-capabilizer
Expand Down
8 changes: 4 additions & 4 deletions examples/https-termination/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ reference a Secret in a different Namespace.

## 1. Deploy NGINX Gateway Fabric

1. Follow the [installation instructions](/docs/installation.md) to deploy NGINX Gateway Fabric.
1. Follow the [installation instructions](https://docs.nginx.com/nginx-gateway-fabric/installation/) to deploy NGINX Gateway Fabric.

1. Save the public IP address of NGINX Gateway Fabric into a shell variable:

Expand Down Expand Up @@ -195,19 +195,19 @@ You can also check the conditions of the Gateway `https` Listener to verify the
Conditions:
Last Transition Time: 2023-06-26T20:23:56Z
Message: Certificate ref to secret certificate/cafe-secret not permitted by any ReferenceGrant
Observed Generation: 2
Observed Generation: 1
Reason: RefNotPermitted
Status: False
Type: Accepted
Last Transition Time: 2023-06-26T20:23:56Z
Message: Certificate ref to secret certificate/cafe-secret not permitted by any ReferenceGrant
Observed Generation: 2
Observed Generation: 1
Reason: RefNotPermitted
Status: False
Type: ResolvedRefs
Last Transition Time: 2023-06-26T20:23:56Z
Message: Certificate ref to secret certificate/cafe-secret not permitted by any ReferenceGrant
Observed Generation: 2
Observed Generation: 1
Reason: Invalid
Status: False
Type: Programmed
Expand Down
8 changes: 4 additions & 4 deletions internal/mode/static/nginx/modules/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion internal/mode/static/nginx/modules/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
},
"devDependencies": {
"c8": "8.0.1",
"prettier": "3.1.0"
"prettier": "3.1.1"
},
"type": "module"
}
2 changes: 1 addition & 1 deletion internal/mode/static/state/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ func (u *validatingUpsertUpdater) Upsert(obj client.Object) {
obj,
apiv1.EventTypeWarning,
"Rejected",
"%s; NGF will delete any existing NGINX configuration that corresponds to the resource",
"%s; NGINX Gateway Fabric will delete any existing NGINX configuration that corresponds to the resource",
err.Error(),
)

Expand Down
2 changes: 1 addition & 1 deletion site/content/how-to/traffic-management/advanced-routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ This request should receive a response from the `coffee-v1` Pod.

```text
Server address: 10.244.0.9:8080
Server name: coffee-v2-68bd55f798-s9z5q
Server name: coffee-v1-76c7c85bbd-cf8nz
```

If we want our request to be routed to `coffee-v2`, then we need to meet the defined conditions. We can include a header:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ spec:
EOF
```

This gateway is associated with the NGINX Gateway Fabric through the **gatewayClassName** field. The default installation of NGINX Gateway Fabric creates a GatewayClass with the name **nginx**. NGINX Gateway Fabric will only configure gateways with a **gatewayClassName** of **nginx** unless you change the name via the `--gatewayclass` [command-line flag](/docs/cli-help.md#static-mode).
This gateway is associated with the NGINX Gateway Fabric through the **gatewayClassName** field. The default installation of NGINX Gateway Fabric creates a GatewayClass with the name **nginx**. NGINX Gateway Fabric will only configure gateways with a **gatewayClassName** of **nginx** unless you change the name via the `--gatewayclass` [command-line flag]({{< relref "/reference/cli-help.md#static-mode" >}}).

We specify a [listener](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1.Listener) on the gateway to open an entry point on the cluster. In this case, since the coffee application accepts HTTP requests, we create an HTTP listener, named **http**, that listens on port 80.

Expand Down
168 changes: 65 additions & 103 deletions site/content/overview/resource-validation.md
Original file line number Diff line number Diff line change
@@ -1,70 +1,23 @@
---
title: "Gateway API Resource Validation"
description: "Learn how NGINX Gateway Fabric validates Gateway API resources."
weight: 800
toc: true
docs: "DOCS-000"
---

## Overview

There are several reasons why NGF validates Gateway API resources:
NGINX Gateway Fabric validates Gateway API resources for several reasons:

- *Robustness*: to gracefully handle invalid resources.
- *Security*: to prevent malicious input from propagating to the NGINX configuration.
- *Correctness*: to conform to the Gateway API specification for handling invalid resources.
- _Robustness_: to gracefully handle invalid resources.
- _Security_: to prevent malicious input from propagating to the NGINX configuration.
- _Correctness_: to conform to the Gateway API specification for handling invalid resources.

Ultimately, the goal is to ensure that NGINX continues to handle traffic even if invalid Gateway API resources were
created.
The process involves four different steps, explained in detail in this document, with the goal of making sure that NGINX continues to handle traffic even if invalid Gateway API resources were created.

A Gateway API resource (a new resource or an update for the existing one) is validated by the following steps:
## Step 1 - OpenAPI Scheme validation by Kubernetes API Server

### For Kubernetes 1.25+

1. OpenAPI schema validation by the Kubernetes API server.
2. CEL validation by the Kubernetes API server.
3. Webhook validation by NGF.
4. Validation by NGF.

### For Kubernetes 1.23 and 1.24

1. OpenAPI schema validation by the Kubernetes API server.
2. Webhook validation by the Gateway API webhook.
3. Webhook validation by NGF.
4. Validation by NGF.

To confirm that a resource is valid and accepted by NGF, check that the `Accepted` condition in the resource status
has the Status field set to `True`. For example, in a status of a valid HTTPRoute, if NGF accepts a parentRef,
the status of that parentRef will look like this:

```text
Status:
Parents:
Conditions:
Last Transition Time: 2023-03-30T23:18:00Z
Message: The route is accepted
Observed Generation: 2
Reason: Accepted
Status: True
Type: Accepted
Controller Name: gateway.nginx.org/nginx-gateway-controller
Parent Ref:
Group: gateway.networking.k8s.io
Kind: Gateway
Name: gateway
Namespace: default
Section Name: http
```

> Make sure the reported observed generation is the same as the resource generation.
The remaining part of this document describes each step in detail with examples of how validation errors are reported.

### Step 1 - OpenAPI Scheme Validation by Kubernetes API Server

The Kubernetes API server validates Gateway API resources against the OpenAPI schema embedded in the Gateway API CRDs.
For example, if you create an HTTPRoute with an invalid hostname `cafe.!@#$%example.com`, the API server will reject it
with the following error:
The Kubernetes API server validates Gateway API resources against the OpenAPI schema embedded in the Gateway API CRDs. For example, if you create an HTTPRoute with an invalid hostname "cafe.!@#$%example.com", the API server will reject it with the following error:

```shell
kubectl apply -f coffee-route.yaml
Expand All @@ -74,53 +27,44 @@ kubectl apply -f coffee-route.yaml
The HTTPRoute "coffee" is invalid: spec.hostnames[0]: Invalid value: "cafe.!@#$%example.com": spec.hostnames[0] in body should match '^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$'
```

> While unlikely, bypassing this validation step is possible if the Gateway API CRDs are modified to remove the validation.
> If this happens, Step 4 will reject any invalid values (from NGINX perspective).
{{< note >}}While unlikely, bypassing this validation step is possible if the Gateway API CRDs are modified to remove the validation. If this happens, Step 4 will reject any invalid values (from NGINX perspective).{{< /note >}}

### Step 2 - For Kubernetes 1.25+ - CEL Validation by Kubernetes API Server
## Step 2 - CEL or Webhook validation by Kubernetes

The Kubernetes API server validates Gateway API resources using CEL validation embedded in the Gateway API CRDs.
It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation.
For example, if you create a Gateway resource with a TCP listener that configures a hostname, the CEL validation will
reject it with the following error:
- **Kubernetes 1.25 and later - CEL validation by Kubernetes API Server**

The Kubernetes API server validates Gateway API resources using CEL validation embedded in the Gateway API CRDs. It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation. For example, if you create a Gateway resource with a TCP listener that configures a hostname, the CEL validation will reject it with the following error:

```shell
kubectl apply -f some-gateway.yaml
```
```shell
kubectl apply -f some-gateway.yaml
```

```text
The Gateway "some-gateway" is invalid: spec.listeners: Invalid value: "array": hostname must not be specified for protocols ['TCP', 'UDP']
```
```text
The Gateway "some-gateway" is invalid: spec.listeners: Invalid value: "array": hostname must not be specified for protocols ['TCP', 'UDP']
```

More information on CEL in Kubernetes can be found [here](https://kubernetes.io/docs/reference/using-api/cel/).
More information on CEL in Kubernetes can be found [here](https://kubernetes.io/docs/reference/using-api/cel/).

### Step 2 - For Kubernetes 1.23 and 1.24 - Webhook Validation by Gateway API Webhook

The Gateway API comes with a validating webhook which is enabled by default in the Gateway API installation manifests.
It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation. For example, if
you create a Gateway resource with a TCP listener that configures a hostname, the webhook will reject it with the
following error:
- **Kubernetes 1.23 and 1.24 - Webhook validation by Gateway API Webhook**

```shell
kubectl apply -f some-gateway.yaml
```
The validating webhook must be [installed for these Kubernetes versions]({{< relref "installation/installing-ngf/helm.md#installing-the-gateway-api-resources" >}}). It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation. For example, if you create a Gateway resource with a TCP listener that configures a hostname, the webhook will reject it with the following error:

```text
Error from server: error when creating "some-gateway.yaml": admission webhook "validate.gateway.networking.k8s.io" denied the request: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP
```
```shell
kubectl apply -f some-gateway.yaml
```

> Bypassing this validation step is possible if the webhook is not running in the cluster.
> If this happens, Step 3 will reject the invalid values.
```text
Error from server: error when creating "some-gateway.yaml": admission webhook "validate.gateway.networking.k8s.io" denied the request: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP
```

### Step 3 - Webhook validation by NGF
{{< note >}}Bypassing this validation step is possible if the webhook is not running in the cluster. If this happens, Step 3 will reject the invalid values.{{< /note >}}

To ensure that the resources are validated with the webhook validation rules, even if the webhook is not running,
NGF performs the same validation. However, NGF performs the validation *after* the Kubernetes API server accepts
the resource.
## Step 3 - Webhook validation by NGINX Gateway Fabric

Below is an example of how NGF rejects an invalid resource (a Gateway resource with a TCP listener that configures a
hostname) with a Kubernetes event:
To ensure that the resources are validated with the webhook validation rules, even if the webhook is not running, NGINX Gateway Fabric performs the same validation. However, NGINX Gateway Fabric performs the validation _after_ the Kubernetes API server accepts the resource.

Below is an example of how NGINX Gateway Fabric rejects an invalid resource (a Gateway resource with a TCP listener that configures a hostname) with a Kubernetes event:

```shell
kubectl describe gateway some-gateway
Expand All @@ -131,28 +75,21 @@ kubectl describe gateway some-gateway
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Rejected 6s nginx-gateway-fabric-nginx the resource failed webhook validation, however the Gateway API webhook failed to reject it with the error; make sure the webhook is installed and running correctly; validation error: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP; NGF will delete any existing NGINX configuration that corresponds to the resource
Warning Rejected 6s nginx-gateway-fabric-nginx the resource failed webhook validation, however the Gateway API webhook failed to reject it with the error; make sure the webhook is installed and running correctly; validation error: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP; NGINX Gateway Fabric will delete any existing NGINX configuration that corresponds to the resource
```

> This validation step always runs and cannot be bypassed.
> NGF will ignore any resources that fail the webhook validation, like in the example above.
> If the resource previously existed, NGF will remove any existing NGINX configuration for that resource.
{{< note >}}This validation step always runs and cannot be bypassed. NGINX Gateway Fabric will ignore any resources that fail the webhook validation, like in the example above. If the resource previously existed, NGINX Gateway Fabric will remove any existing NGINX configuration for that resource.{{< /note >}}

### Step 4 - Validation by NGF
## Step 4 - Validation by NGINX Gateway Fabric

This step catches the following cases of invalid values:

- Valid values from the Gateway API perspective but not supported by NGF yet. For example, a feature in an
HTTPRoute routing rule. Note: for the list of supported features,
see [Gateway API Compatibility](gateway-api-compatibility.md) doc.
- Valid values from the Gateway API perspective, but invalid for NGINX, because NGINX has stricter validation
requirements for certain fields. Such values will cause NGINX to fail to reload or operate erroneously.
- Invalid values (both from the Gateway API and NGINX perspectives) that were not rejected because Step 1 was bypassed.
Similarly to the previous case, such values will cause NGINX to fail to reload or operate erroneously.
- Malicious values that inject unrestricted NGINX config into the NGINX configuration (similar to an SQL injection
attack).
- Valid values from the Gateway API perspective but not supported by NGINX Gateway Fabric yet. For example, a feature in an HTTPRoute routing rule. For the list of supported features see [Gateway API Compatibility](gateway-api-compatibility.md) doc.
- Valid values from the Gateway API perspective, but invalid for NGINX, because NGINX has stricter validation requirements for certain fields. These values will cause NGINX to fail to reload or operate erroneously.
- Invalid values (both from the Gateway API and NGINX perspectives) that were not rejected because Step 1 was bypassed. Similar to the previous case, these values will cause NGINX to fail to reload or operate erroneously.
- Malicious values that inject unrestricted NGINX config into the NGINX configuration (similar to an SQL injection attack).

Below is an example of how NGF rejects an invalid resource. The validation error is reported via the status:
Below is an example of how NGINX Gateway Fabric rejects an invalid resource. The validation error is reported via the status:

```shell
kubectl describe httproutes.gateway.networking.k8s.io coffee
Expand All @@ -178,4 +115,29 @@ Status:
Section Name: http
```

> This validation step always runs and cannot be bypassed.
{{< note >}}This validation step always runs and cannot be bypassed.{{< /note >}}

## Confirm validation

To confirm that a resource is valid and accepted by NGINX Gateway Fabric, check that the **Accepted** condition in the resource status has the Status field set to **True**. For example, in a status of a valid HTTPRoute, if NGINX Gateway Fabric accepts a parentRef, the status of that parentRef will look like this:

```text
Status:
Parents:
Conditions:
Last Transition Time: 2023-03-30T23:18:00Z
Message: The route is accepted
Observed Generation: 2
Reason: Accepted
Status: True
Type: Accepted
Controller Name: gateway.nginx.org/nginx-gateway-controller
Parent Ref:
Group: gateway.networking.k8s.io
Kind: Gateway
Name: gateway
Namespace: default
Section Name: http
```

{{< note>}}Make sure the reported observed generation is the same as the resource generation.{{< /note >}}
2 changes: 1 addition & 1 deletion site/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ module github.com/nginxinc/nginx-gateway-fabric/site

go 1.21

require github.com/nginxinc/nginx-hugo-theme v0.40.0 // indirect
require github.com/nginxinc/nginx-hugo-theme v0.40.1 // indirect
2 changes: 2 additions & 0 deletions site/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@ github.com/nginxinc/nginx-hugo-theme v0.39.0 h1:P1hOPpityVUOM5OyIpQZa1UJyuUunGSm
github.com/nginxinc/nginx-hugo-theme v0.39.0/go.mod h1:DPNgSS5QYxkjH/BfH4uPDiTfODqWJ50NKZdorguom8M=
github.com/nginxinc/nginx-hugo-theme v0.40.0 h1:YP0I0+bRKcJ5WEb1s/OWcnlcvNvIcKscagJkCzsa+Vs=
github.com/nginxinc/nginx-hugo-theme v0.40.0/go.mod h1:DPNgSS5QYxkjH/BfH4uPDiTfODqWJ50NKZdorguom8M=
github.com/nginxinc/nginx-hugo-theme v0.40.1 h1:1Q94uFYegNvjvwDV1py9VlYmh62AF1gh1oPGqjNmtis=
github.com/nginxinc/nginx-hugo-theme v0.40.1/go.mod h1:DPNgSS5QYxkjH/BfH4uPDiTfODqWJ50NKZdorguom8M=
3 changes: 2 additions & 1 deletion site/mdlint_conf.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@
"siblings_only": true
},
"MD046": false,
"MD001": false
"MD001": false,
"MD049": false
}
Loading

0 comments on commit 50be6f4

Please sign in to comment.