Skip to content

Commit

Permalink
Adding documentation for policy attachment model
Browse files Browse the repository at this point in the history
  • Loading branch information
robscott committed Jul 26, 2021
1 parent d8d28f4 commit dd8737b
Show file tree
Hide file tree
Showing 4 changed files with 366 additions and 1 deletion.
56 changes: 56 additions & 0 deletions apis/v1alpha2/policy_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
/*
Copyright 2021 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha2

// PolicyTargetReference identifies an API object to apply policy to.
type PolicyTargetReference struct {
// Group is the group of the target resource.
//
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=253
Group string `json:"group"`

// Kind is kind of the target resource.
//
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=253
Kind string `json:"kind"`

// Name is the name of the target resource.
//
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=253
Name string `json:"name"`

// Namespace is the namespace of the referent. When unspecified, the local
// namespace is inferred. Even when policy targets a resource in a different
// namespace, it may only apply to traffic originating from the same
// namespace as the policy.
//
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=253
// +optional
Namespace string `json:"namespace,omitempty"`

// ClassName is the name of the class this policy should apply to. When
// unspecified, the policy will apply to all classes that support it.
//
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=253
// +optional
ClassName string `json:"className,omitempty"`
}
15 changes: 15 additions & 0 deletions apis/v1alpha2/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion site-src/geps/gep-713.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# GEP-713: Policy Attachment

* Issue: [#713](https://github.com/kubernetes-sigs/gateway-api/issues/713)
* Status: Implementable
* Status: Implemented

## TLDR

Expand Down
294 changes: 294 additions & 0 deletions site-src/v1alpha2/references/policy-attachment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,294 @@
# Policy Attachment

Policies attached to Gateway API resources and implementations should use the
following approach to ensure consistency across implementations of the API.
There are three primary components of this pattern:

* A shared means of attaching policy to resources.
* Support for configuring both default and override values within policy
resources.
* A shared hierarchy to illustrate how default and override values should
interact.

This kind of standardization will not only enable consistent patterns, it will
enable tooling such as kubectl plugins to be able to visualize all policies that
have been applied to a given resource.

## Policy Attachment for Ingress
Attaching policy to Gateway resources for ingress use cases is relatively
straightforward. A policy can reference the resource it wants to apply to.
Access is granted with RBAC - anyone that has access to create a RetryPolicy in
a given namespace can attach it to any resource within that namespace.

![Simple Ingress Example](/geps/images/713-ingress-simple.png)

To build on that example, it’s possible to attach policies to more resources.
Each policy applies to the referenced resource and everything below it in terms
of hierarchy. Although this example is likely more complex than many real world
use cases, it helps demonstrate how policy attachment can work across
namespaces.

![Complex Ingress Example](/geps/images/713-ingress-complex.png)

## Policy Attachment for Mesh
Although there is a great deal of overlap between ingress and mesh use cases,
mesh enables more complex policy attachment scenarios. For example, you may want
to apply policy to requests from a specific namespace to a backend in another
namespace.

![Simple Mesh Example](/geps/images/713-mesh-simple.png)

Policy attachment can be quite simple with mesh. Policy can be applied to any
resource in any namespace but it can only apply to requests from the same
namespace if the target is in a different namespace.

At the other extreme, policy can be used to apply to requests from a specific
workload to a backend in another namespace. A route can be used to intercept
these requests and split them between different backends (foo-a and foo-b in
this case).

![Complex Mesh Example](/geps/images/713-mesh-complex.png)

## Target Reference API

Each Policy resource MUST include a single `targetRef` field. It MUST not
target more than one resource at a time, but it can be used to target larger
resources such as Gateways or Namespaces that may apply to multiple child
resources.

The `targetRef` field MUST be an exact replica of the `PolicyTargetReference`
struct included in the Gateway API. Where possible, we recommend using that
struct directly instead of duplicating the type.

### Policy Boilerplate
The following structure can be used as a starting point for any Policy resource
using this API pattern.

```go
// ACMEServicePolicy provides a way to apply Service policy configuration with
// the ACME implementation of the Gateway API.
type ACMEServicePolicy struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

// Spec defines the desired state of ACMEServicePolicy.
Spec ACMEServicePolicySpec `json:"spec"`

// Status defines the current state of ACMEServicePolicy.
Status ACMEServicePolicyStatus `json:"status,omitempty"`
}

// ACMEServicePolicySpec defines the desired state of ACMEServicePolicy.
type ACMEServicePolicySpec struct {
// TargetRef identifies an API object to apply policy to.
TargetRef gatewayv1a2.PolicyTargetReference `json:"targetRef"`

// Override defines policy configuration that should override policy
// configuration attached below the targeted resource in the hierarchy.
// +optional
Override *ACMEPolicyConfig `json:"override,omitempty"`

// Default defines default policy configuration for the targeted resource.
// +optional
Default *ACMEPolicyConfig `json:"default,omitempty"`
}

// ACMEPolicyConfig contains ACME policy configuration.
type ACMEPolicyConfig struct {
// Add configurable policy here
}

// ACMEServicePolicyStatus defines the observed state of ACMEServicePolicy.
type ACMEServicePolicyStatus struct {
// Conditions describe the current conditions of the ACMEServicePolicy.
//
// +optional
// +listType=map
// +listMapKey=type
// +kubebuilder:validation:MaxItems=8
Conditions []metav1.Condition `json:"conditions,omitempty"`
}
```

### Hierarchy
Each policy MAY include default or override values. Default values are given
precedence from the bottom up, while override values are top down. That means
that a default attached to a Backend will have the highest precedence among
default values while an override value attached to a GatewayClass will have the
highest precedence overall.

![Ingress and Sidecar Hierarchy](/geps/images/713-hierarchy.png)

To illustrate this, consider 3 resources with the following hierarchy:
A > B > C. When attaching the concept of defaults and overrides to that, the
hierarchy would be expanded to this:

A override > B override > C override > C default > B default > A default.

Note that the hierarchy is reversed for defaults. The rationale here is that
overrides usually need to be enforced top down while defaults should apply to
the lowest resource first. For example, if an admin needs to attach required
policy, they can attach it as an override to a Gateway. That would have
precedence over Routes and Services below it. On the other hand, an app owner
may want to set a default timeout for their Service. That would have precedence
over defaults attached at higher levels such as Route or Gateway.

If using defaults and overrides, each policy resource MUST include 2 structs
within the spec. One with override values and the other with default values.

In the following example, the policy attached to the Gateway requires cdn to
be enabled and provides some default configuration for that. The policy attached
to the Route changes the value for one of those fields (includeQueryString).

```yaml
kind: GKEServicePolicy # Example of implementation specific policy name
spec:
override:
cdn:
enabled: true
default:
cdn:
cachePolicy:
includeHost: true
includeProtocol: true
includeQueryString: true
targetRef:
kind: Gateway
name: example
---
kind: GKEServicePolicy
spec:
default:
cdn:
cachePolicy:
includeQueryString: false
targetRef:
kind: HTTPRoute
name: example
```
In this final example, we can see how the override attached to the Gateway has
precedence over the default drainTimeout value attached to the Route. At the
same time, we can see that the default connectionTimeout attached to the Route
has precedence over the default attached to the Gateway.
![Hierarchical Policy Example](/geps/images/713-policy-hierarchy.png)
#### Supported Resources
It is important to note that not every implementation will be able to support
policy attachment to each resource described in the hierarchy above. When that
is the case, implementations MUST clearly document which resources a policy may
be attached to.
#### Attaching Policy to GatewayClass
GatewayClass may be the trickiest resource to attach policy to. Policy
attachment relies on the policy being defined within the same scope as the
target. This ensures that only users with write access to a policy resource in a
given scope will be able to modify policy at that level. Since GatewayClass is a
cluster scoped resource, this means that any policy attached to it must also be
cluster scoped.
GatewayClass parameters provide an alternative to policy attachment that may be
easier for some implementations to support. These parameters can similarly be
used to set defaults and requirements for an entire GatewayClass.
### Targeting External Services
In some cases (likely limited to mesh) we may want to apply policies to requests
to external services. To accomplish this, implementations can choose to support
a refernce to a virtual resource type:
```yaml
kind: RetryPolicy
apiVersion: networking.acme.io/v1alpha1
metadata:
name: foo
spec:
default:
maxRetries: 5
targetRef:
group: networking.acme.io
kind: ExternalService
name: foo.com
```
### Conflict Resolution
It is possible for multiple policies to target the same resource. When this
happens, merging is the preferred outcome. If multiple policy resources target
the same resource _and_ have an identical field specified with different values,
precedence MUST be determined in order of the following criteria, continuing on
ties:
* The oldest Policy based on creation timestamp. For example, a Policy with a
creation timestamp of "2021-07-15 01:02:03" is given precedence over a Policy
with a creation timestamp of "2021-07-15 01:02:04".
* The Policy appearing first in alphabetical order by "<namespace>/<name>". For
example, foo/bar is given precedence over foo/baz.
For a better user experience, a validating webhook can be implemented to prevent
these kinds of conflicts all together.
### Kubectl Plugin
To help improve UX and standardization, a kubectl plugin will be developed that
will be capable of describing the computed sum of policy that applies to a given
resource, including policies applied to parent resources.
Each Policy CRD that wants to be supported by this plugin will need to follow
the API structure defined above and add a `gateway.networking.k8s.io/policy:
true` label to the CRD.

### Status
In the future, we may consider adding a new `Policies` field to status on
Gateways and Routes. This would be a list of `PolicyTargetReference` structs
with the fields instead used to refer to the Policy resource that has been
applied.

Unfortunately, this may create more confusion than it is worth, here are some of
the key concerns:

* When multiple controllers are implementing the same Route and recognize a
policy, it would be difficult to determine which controller should be
responsible for adding that policy reference to status.
* For this to be somewhat scalable, we'd need to limit the status entries to
policies that had been directly applied to the resource. This could get
confusing as it would not provide any insight into policies attached above or
below.
* Since we only control some of the resources a policy might be attached to,
adding policies to status would only be possible on Gateway API resources, not
Services or other kinds of backends.

Although these concerns are not unsolvable, they lead to the conclusion that
a Kubectl plugin should be our primary approach to providing visibility here,
with a possibility of adding policies to status at a later point.

### Interaction with Custom Filters and other extension points
There are multiple methods of custom extension in the Gateway API. Policy
attachment and custom Route filters are two of these. Policy attachment is
designed to provide arbitrary configuration fields that decorate Gateway API
resources. Route filters provide custom request/response filters embedded inside
Route resources. Both are extension methods for fields that cannot easily be
standardized as core or extended fields of the Gateway API. The following
guidance should be considered when introducing a custom field into any Gateway
controller implementation:

1. For any given field that a Gateway controller implementation needs, the
possibility of using core or extended should always be considered before
using custom policy resources. This is encouraged to promote standardization
and, over time, to absorb capabilities into the API as first class fields,
which offer a more streamlined UX than custom policy attachment.

2. Although it's possible that arbitrary fields could be supported by custom
policy, custom route filters, and core/extended fields concurrently, it is
strongly recommended that implementations not use multiple mechanisms for
representing the same fields. A given field should only be supported through
a single extension method. An example of potential conflict is policy
precedence and structured hierarchy, which only applies to custom policies.
Allowing a field to exist in custom policies and also other areas of the API,
which are not part of the structured hierarchy, breaks the precedence model.
Note that this guidance may change in the future as we gain a better
understanding for extension mechanisms of the Gateway API can interoperate.

### Conformance Level
This policy attachment pattern is associated with an "EXTENDED" conformance
level. The implementations that support this policy attachment model will have
the same behavior and semantics, although they may not be able to support
attachment of all types of policy at all potential attachment points.

0 comments on commit dd8737b

Please sign in to comment.