Skip to content

Commit

Permalink
feat: support additional resources across tenant namespaces
Browse files Browse the repository at this point in the history
This attempts to address  projectcapsule#525 which is related to projectcapsule#416.

problem this commit is trying to solve
----

```
Alice would like to create several resources on each of their Namespaces.
Upon the creation of each Namespace, they have to create the desired resources on each Namespace
```

On the above linked tickets there are two proposed approaches

approach 01
----

Create a new resource `TenantResource`  something like this

```yaml
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: database
  namespace: solar-management
spec:
  resyncPeriod: 60s
  additionalResources:
    namespaceSelector:
      matchLabels:
        tier: one
    items:
    - apiVersion: presslabs.io/v1beta1
      kind: MySQL
      spec:
        foo: bar
  clusterRoles:
    namespaceSelector:
      matchLabels:
        tier: one
    items:
      - name: database-admin
        subjects:
        - kind: Group
          name: developers
```

approach 02
-----

Extend `Tenant` to  support addtional resources

```yaml
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
  name: gas
spec:
  additionalResources:
    -  apiVersion: "cilium.io/v2"
       kind: CiliumNetworkPolicy
       metadata:
         name: "l3-rule"
      spec:
        endpointSelector:
          matchLabels:
             role: backend
       ingress:
       - fromEndpoints:
         - matchLabels:
             role: frontend
```

This commit implements approach `02` due to the following reasons

- The namespaces belong to the tenant already, extra `TenantResource` seems redundant given that the lifecycle of the additional resources are tied to the `Tenant`.

How does the crd look like now ?
----

```yaml
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
  name: oil
spec:
  resyncPeriod: 60s
  additionalResources:
    namespaceSelector:
      matchLabels:
        tier: one
    items:
      - |
        apiVersion: v1
        kind: Pod
        metadata:
          name: nginx
          labels:
            app.kubernetes.io/name: proxy
        spec:
          containers:
            - name: nginx
              image: nginx:11.14.2
              ports:
                - containerPort: 80
                  name: http-web-svc
      - |
        apiVersion: v1
        kind: Service
        metadata:
          name: nginx-service
          labels:
            app.kubernetes.io/name: proxy
        spec:
          selector:
            app.kubernetes.io/name: proxy
          ports:
            - name: name-of-service-port
              protocol: TCP
              port: 80
              targetPort: http-web-svc
  owners:
    - name: alice
      kind: User
```

The difference with the proposed crd is, items are strings of k8s objects in yaml
format. I ran through decoding issues when I tried to use `Unstructured`  so I
decided to settle on strings instead.

How it works ?
----

We search for namespaces  specified by `namespaceSelector` that are owned by the
tenant. For each matched namespace, apply all resources specified in `additionalResources.items`
on any error, reschedule next reconciliation by `resyncPeriod`.

What is missing ?
----

- [ ] Tests
- [ ] What happens when a tenant is deleted ?
- [ ] What happens when a tenant is deleted ?
- [ ] Does `additionalRoleBindings` cover for  `clusterRoles` defined in approach 01?

I will wait for feedback/discussion on how to proceed from here.
  • Loading branch information
gernest committed Apr 18, 2022
1 parent a3495cf commit 6423628
Show file tree
Hide file tree
Showing 11 changed files with 1,404 additions and 1,168 deletions.
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -214,4 +214,4 @@ e2e/%: ginkgo
capsule \
./charts/capsule
$(GINKGO) -v -tags e2e ./e2e
kind delete cluster --name capsule
kind delete cluster --name capsule
11 changes: 11 additions & 0 deletions api/v1beta1/additional_resources.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package v1beta1

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

type AdditionalResources struct {
NamespaceSelector metav1.LabelSelector `json:"namespaceSelector"`
// list of k8s resources in yaml format
Items []string `json:"items"`
}
7 changes: 7 additions & 0 deletions api/v1beta1/tenant_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ import (

// TenantSpec defines the desired state of Tenant
type TenantSpec struct {
// Duration for reconciliation loop to poll and ensure AdditionalResources are
// in correct state.
//+kubebuilder:default="60s"
ResyncPeriod *metav1.Duration `json:"resyncPeriod,omitempty"`
// AdditionalResources specifies additional resources that will be created with
// the tenant
AdditionalResources *AdditionalResources `json:"additionalResources,omitempty"`
// Specifies the owners of the Tenant. Mandatory.
Owners OwnerListSpec `json:"owners"`
// Specifies options for the Namespaces, such as additional metadata or maximum number of namespaces allowed for that Tenant. Once the namespace quota assigned to the Tenant has been reached, the Tenant owner cannot create further namespaces. Optional.
Expand Down
32 changes: 32 additions & 0 deletions api/v1beta1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

68 changes: 34 additions & 34 deletions charts/capsule/crds/capsuleconfiguration-crd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,43 +14,43 @@ spec:
singular: capsuleconfiguration
scope: Cluster
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: CapsuleConfiguration is the Schema for the Capsule configuration API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: CapsuleConfigurationSpec defines the Capsule configuration
properties:
forceTenantPrefix:
default: false
description: Enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash. This is useful to avoid Namespace name collision in a public CaaS environment.
type: boolean
protectedNamespaceRegex:
description: Disallow creation of namespaces, whose name matches this regexp
- name: v1alpha1
schema:
openAPIV3Schema:
description: CapsuleConfiguration is the Schema for the Capsule configuration API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: CapsuleConfigurationSpec defines the Capsule configuration
properties:
forceTenantPrefix:
default: false
description: Enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash. This is useful to avoid Namespace name collision in a public CaaS environment.
type: boolean
protectedNamespaceRegex:
description: Disallow creation of namespaces, whose name matches this regexp
type: string
userGroups:
default:
- capsule.clastix.io
description: Names of the groups for Capsule users.
items:
type: string
userGroups:
default:
- capsule.clastix.io
description: Names of the groups for Capsule users.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
storedVersions: []
Loading

0 comments on commit 6423628

Please sign in to comment.