-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add docs for limits/requests, affinity, reword PDB #393
Merged
Merged
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
bfba484
chore: Rename 03-production-deployments.md
viccuad 8ec09ef
docs: Add limits, requests, affinity, reword PDB
viccuad 81a7294
Apply suggestions from code review
viccuad ff042a6
Apply suggestions from code review
viccuad 543a671
Apply suggestions from code review
viccuad 61a6518
Apply suggestions from code review
viccuad File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
209 changes: 209 additions & 0 deletions
209
docs/howtos/policy-servers/03-production-deployments.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,209 @@ | ||
--- | ||
sidebar_label: Production deployments | ||
title: Configuring PolicyServers for production | ||
description: Configuring PolicyServers for production | ||
keywords: | ||
[ | ||
kubewarden, | ||
kubernetes, | ||
policyservers, | ||
production, | ||
poddisruptionbudget, | ||
affinity, | ||
limits, | ||
] | ||
doc-persona: [kubewarden-operator, kubewarden-integrator] | ||
doc-type: [howto] | ||
doc-topic: | ||
[ | ||
operator-manual, | ||
policy-servers, | ||
production, | ||
poddisruptionbudget, | ||
affinity, | ||
limits, | ||
] | ||
--- | ||
|
||
PolicyServers are critical to the cluster. Reliability of them is important as | ||
they process Admission Requests destined for the Kube API via the Validating and | ||
viccuad marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Mutating Webhooks. | ||
|
||
As with other Dynamic Admission Controllers, this process happens before | ||
requests reach the Kube API server. Latency or service delays by | ||
viccuad marked this conversation as resolved.
Show resolved
Hide resolved
|
||
the Dynamic Admission Controller may introduce cluster inconsistency, | ||
Denial of Service, or deadlock. | ||
|
||
Kubewarden provides several ways to increase the reliability of PolicyServers. | ||
Production deployments can vary a great deal, it is up to the operator to configure the deployment for their needs. | ||
|
||
# PodDistruptionBudgets | ||
|
||
The Kubewarden controller can create a | ||
[PodDisruptionBudget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/) | ||
(PDB) for the policy-server Pods. This controls the range of policy-server | ||
Pod replicas associated with the PolicyServer, ensuring high availability | ||
and controlled eviction in case of node maintenance, scaling operations or | ||
cluster upgrades. | ||
|
||
This is achieved by setting `spec.minAvailable`, or `spec.maxUnavailable` of the | ||
PolicyServer resource: | ||
|
||
- `minAvailable`: specifies the minimum number of policy-server Pods | ||
that must be available at all times. Can be an integer or a percentage. | ||
|
||
Useful for maintaining the operational integrity of the PolicyServer, | ||
ensuring that policies are continously enforced without interruption. | ||
|
||
- `maxUnavailable`: specifies the maximum number of policy-server Pods that can | ||
be unavailable at any given time. Can be an integer or a percentage. | ||
|
||
Useful for performing rolling updates or partial maintenance without fully | ||
halting the policy enforcement mechanism. | ||
|
||
:::note | ||
You can specify only one of `maxUnavailable` and `minAvailable`. | ||
::: | ||
|
||
## Configuring minAvailable or maxUnavailable | ||
|
||
Examples: | ||
|
||
```yaml | ||
apiVersion: policies.kubewarden.io/v1 | ||
kind: PolicyServer | ||
metadata: | ||
name: your-policy-server | ||
spec: | ||
# Other configuration fields | ||
minAvailable: 2 # ensure at least two policy-server Pods are available at all times | ||
``` | ||
|
||
```yaml | ||
apiVersion: policies.kubewarden.io/v1 | ||
kind: PolicyServer | ||
metadata: | ||
name: your-policy-server | ||
spec: | ||
# Other configuration fields | ||
maxUnavailable: "30%" # ensure no more than 30% of policy-server Pods are unavailable at all times | ||
``` | ||
|
||
# Affinity / Anti-affinity | ||
|
||
The Kubewarden controller can set the affinity of policy-server Pods. This | ||
allows constraint of Pods to specific nodes, or Pods against other Pods. For | ||
more information on Affinity, see the [Kubernetes | ||
docs](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). | ||
|
||
Kubernetes affinity configuration allows constraining Pods to nodes (via | ||
`spec.affinity.nodeAffinity`) or constraining Pods with regards to other Pods | ||
(via `spec.affinity.podAffinity`). Affinity can be set as a soft constraint | ||
(with `preferredDuringSchedulingIgnoredDuringExecution`) or a hard one (with | ||
`requiredDuringSchedulingIgnoredDuringExecution`). | ||
|
||
Affinity / anti-affinity matches against specific labels, be it nodes' labels | ||
(e.g: `topology.kubernetes.io/zone` set to `antarctica-east1`) or Pods labels. | ||
Pods created from PolicyServer definitions have a label | ||
`kubewarden/policy-server` set to the name of the PolicyServer. (e.g: | ||
`kubewarden/policy-server: default`). | ||
|
||
:::note | ||
Inter-pod affinity/anti-affinity require substantial amounts of processing and | ||
are not recommended in clusters larger than several hundred nodes. | ||
::: | ||
|
||
To configure affinity for a PolicyServer, set its `spec.affinity` field. This | ||
field accepts a YAML object matching the contents of a Pod's `spec.affinity`. | ||
|
||
## Configuring Affinity / Anti-affinity | ||
|
||
Example: Prefer to spread the PolicyServer Pods across zones and hostnames | ||
|
||
```yaml | ||
apiVersion: policies.kubewarden.io/v1 | ||
kind: PolicyServer | ||
metadata: | ||
name: your-policy-server | ||
spec: | ||
# Other configuration fields | ||
affinity: | ||
podAntiAffinity: | ||
preferredDuringSchedulingIgnoredDuringExecution: | ||
- labelSelector: | ||
matchExpressions: | ||
- key: kubewarden/policy-server | ||
operator: In | ||
values: | ||
- your-policy-server | ||
topologyKey: topology.kubernetes.io/zone | ||
- labelSelector: | ||
matchExpressions: | ||
- key: kubewarden/policy-server | ||
operator: In | ||
values: | ||
- your-policy-server | ||
topologyKey: kubernetes.io/hostname | ||
``` | ||
|
||
Example: Only schedule PolicyServer pods in control-plane nodes | ||
|
||
```yaml | ||
apiVersion: policies.kubewarden.io/v1 | ||
kind: PolicyServer | ||
metadata: | ||
name: your-policy-server | ||
spec: | ||
# Other configuration fields | ||
affinity: | ||
podAffinity: | ||
requiredDuringSchedulingIgnoredDuringExecution: | ||
- labelSelector: | ||
matchExpressions: | ||
- key: kubewarden/policy-server | ||
operator: In | ||
values: | ||
- your-policy-server | ||
topologyKey: node-role.kubernetes.io/control-plane | ||
``` | ||
|
||
# Limits and Requests | ||
|
||
The Kubewarden controller can set the resource limits and requests of | ||
policy-server Pods. This specifies how much of each resource each of the | ||
containers associated with the policy-server Pods needs. For PolicyServers, | ||
only `cpu` and `memory` resources are relevant. See the [Kubernetes | ||
docs](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#resource-units-in-kubernetes) | ||
on resource units for more information. | ||
|
||
This is achieved by setting the following PolicyServer resource fields: | ||
|
||
- `spec.limits`: Limits on resources, enforced by the container runtime. | ||
Different runtimes can have different ways to implement the restrictions. | ||
- `spec.requests`: Amount of resources to reserve for each container. It is | ||
jhkrug marked this conversation as resolved.
Show resolved
Hide resolved
|
||
possible and allowed for a container to use more resource than it's `request`. | ||
|
||
If omitted, it defaults to `spec.limits` if that is set (unless | ||
`spec.requests` of containers is set to some defaults via an admission | ||
mechanism). | ||
|
||
:::note | ||
Undercommitting resources of PolicyServers may cause realibility issues in the | ||
cluster. | ||
::: | ||
|
||
## Configuring Limits and Requests | ||
|
||
Example: Set hard limits for each policy-server container | ||
|
||
```yaml | ||
apiVersion: policies.kubewarden.io/v1 | ||
kind: PolicyServer | ||
metadata: | ||
name: your-policy-server | ||
spec: | ||
# Other configuration fields | ||
limits: | ||
cpu: 500m | ||
memory: 1Gi | ||
``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should have some guidelines about how to spell certain pieces of our tech stack. Do we have to use
PolicyServer
orPolicy Server
?It would be even better to have some keywords that automatically expand to the right term. For example,
&policyServer
gets expanded to bePolicyServer
at rendering time. I don't know if docusaurus has something for thatCC @jhkrug
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we have agreed on PolicyServer, where talking about the component in general. As per the Kubernetes docs style guide. And yes, I can do something about variables in DS. I had to do it for Longhorn. It would be something like
[[< policy-server-component >]]
expands toPolicyServer
, etc.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Opened #394 to not forget about it, please feel free to edit it.