From 0b8b1dfebd886ebfb9fe1dd0fe61dcc634848a58 Mon Sep 17 00:00:00 2001
From: Dario Tranchitella <dario@tranchitella.eu>
Date: Tue, 25 May 2021 17:03:40 +0200
Subject: [PATCH] docs: Pod Priority Class enforcement using Tenant annotations

---
 docs/index.md                                 |  1 +
 docs/operator/use-cases/overview.md           |  2 +-
 docs/operator/use-cases/pod-priority-class.md | 33 +++++++++++++++++++
 3 files changed, 35 insertions(+), 1 deletion(-)
 create mode 100644 docs/operator/use-cases/pod-priority-class.md

diff --git a/docs/index.md b/docs/index.md
index 845de881c..a5f411d35 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -35,6 +35,7 @@ docs
         ├── onboarding.md
         ├── overview.md
         ├── permissions.md
+        ├── pod-priority-class.md
         ├── pod-security-policies.md
         ├── resources-quota-limits.md
         ├── storage-classes.md
diff --git a/docs/operator/use-cases/overview.md b/docs/operator/use-cases/overview.md
index f5ffe4292..12c33e865 100644
--- a/docs/operator/use-cases/overview.md
+++ b/docs/operator/use-cases/overview.md
@@ -25,6 +25,7 @@ Bill, at Acme Corp. can use Capsule to address any of the following scenarios:
 * [Create Namespaces](./create-namespaces.md)
 * [Assign Permissions](./permissions.md)
 * [Enforce Resources Quotas and Limits](./resources-quota-limits.md)
+* [Enforce Pod Priority Classes](./pod-priority-class.md)
 * [Assign specific Node Pools](./nodes-pool.md)
 * [Assign Ingress Classes](./ingress-classes.md)
 * [Assign Ingress Hostnames](./ingress-hostnames.md)
@@ -34,7 +35,6 @@ Bill, at Acme Corp. can use Capsule to address any of the following scenarios:
 * [Assign Pod Security Policies](./pod-security-policies.md)
 * [Create Custom Resources](./custom-resources.md)
 * [Taint Namespaces](./taint-namespaces.md)
-
 * [Assign multiple Tenants to an owner](./multiple-tenants.md)
 
 > NB: as we improve Capsule, more use cases about multi-tenancy and cluster governance will be covered.
diff --git a/docs/operator/use-cases/pod-priority-class.md b/docs/operator/use-cases/pod-priority-class.md
new file mode 100644
index 000000000..3bdb73724
--- /dev/null
+++ b/docs/operator/use-cases/pod-priority-class.md
@@ -0,0 +1,33 @@
+# Enforcing Pod Priority Classes
+
+> Pods can have priority. Priority indicates the importance of a Pod relative to other Pods.
+> If a Pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority Pods to make scheduling of the pending Pod possible.
+> 
+> [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/) 
+
+In a multi-tenant cluster where not all users are trusted, a tenant owner could create Pods at the highest possible priorities, causing other Pods to be evicted/not get scheduled.
+
+At the current state, Capsule doesn't have, yet, a CRD key to handle the enforced [Priority Class](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass).
+
+Enforcement is feasible using the Tenant's annotations field, as following:
+
+```yaml
+apiVersion: capsule.clastix.io/v1alpha1
+kind: Tenant
+metadata:
+  name: oil
+  annotations:
+    priorityclass.capsule.clastix.io/allowed: default
+    priorityclass.capsule.clastix.io/allowed-regex: "^tier-.*$"
+spec:
+  owner:
+    kind: User
+    name: alice
+```
+
+With the said Tenant specification Alice can create Pod resource if `spec.priorityClassName` equals to:
+
+- `default`, as mentioned in the annotation `priorityclass.capsule.clastix.io/allowed`
+- `tier-gold`, `tier-silver`, or `tier-bronze`, since these compile the regex declared in the annotation `priorityclass.capsule.clastix.io/allowed-regex`
+
+If a Pod is going to use a non-allowed _Priority Class_, it will be rejected by the Validation Webhook enforcing it.