-
Notifications
You must be signed in to change notification settings - Fork 616
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refer to authorisation model in RFC-0001
Signed-off-by: Stefan Prodan <[email protected]>
- Loading branch information
1 parent
dc4a670
commit f884db8
Showing
1 changed file
with
29 additions
and
95 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,10 @@ | ||
| # RFC-0001 Flux Multi-Tenancy | ||
| # RFC-0004 Flux Multi-Tenancy | ||
|
|
||
| **Status:** provisional | ||
|
|
||
| **Creation date:** 2021-11-15 | ||
|
|
||
| **Last update:** 2021-12-17 | ||
|
|
||
| ## Summary | ||
|
|
||
|
|
@@ -10,7 +16,8 @@ models for multi-tenancy, and gives reference implementations for those models. | |
| To this point, the Flux project has provided [examples of | ||
| multi-tenancy](https://github.com/fluxcd/flux2-multi-tenancy/tree/v0.1.0), but not explained exactly | ||
| how they relate to Flux's authorisation model. This RFC explains two multi-tenancy implementations, | ||
| their security properties, and how they are implemented within the authorisation model. | ||
| their security properties, and how they are implemented within the authorisation model | ||
| as defined in [RFC-0001](https://github.com/fluxcd/flux2/pull/2212). | ||
|
|
||
| ### Goals | ||
|
|
||
|
|
@@ -54,14 +61,18 @@ to a cluster. These controllers are subject to authorisation on two counts: | |
| - when accessing Kubernetes resources that are needed for a | ||
| particular "apply" operation -- for example, a secret referenced in | ||
| the field `.spec.valuesFrom` in a `HelmRelease`; | ||
| - when creating, updating and deleting Kubernetes resources in the process of applying a piece of | ||
| configuration. | ||
| - when creating, watching, updating and deleting Kubernetes resources | ||
| in the process of applying a piece of configuration. | ||
|
|
||
| To give users control over this authorisation, these two controllers will _impersonate_ (assume the | ||
| identity of) a service account mentioned in the apply specification (e.g., [the field | ||
| `.spec.serviceAccountName` in a `Kustomization` object][serviceAccountName]) for both accessing | ||
| resources and applying configuration. This lets a user constrain the operations mentioned above with | ||
| RBAC. | ||
| identity of) a service account mentioned in the apply specification (e.g., the field | ||
| `.spec.serviceAccountName` in a [`Kustomization` object][kcsa] | ||
| or in a [`HelmRelease` object][hcsa]) for both accessing resources and applying configuration. | ||
| This lets a user constrain the operations mentioned above with RBAC. | ||
|
|
||
| As stated in [RFC-0003](https://github.com/fluxcd/flux2/pull/2093), | ||
| the platform admins can configure Flux to enforce service account impersonation | ||
| by setting a default service account name when `.spec.serviceAccountName` is not specified. | ||
|
|
||
| #### Remote apply | ||
|
|
||
|
|
@@ -72,7 +83,8 @@ the client used to apply the specified set of configuration. The effect of this | |
| configuration will be applied as the user given in the kubeconfig; often this is a user with the | ||
| `cluster-admin` role bound to it, but not necessarily so. | ||
|
|
||
| [serviceAccountName]: https://fluxcd.io/docs/components/kustomize/api/#kustomize.toolkit.fluxcd.io/v1beta2.KustomizationSpec | ||
| [kcsa]: https://fluxcd.io/docs/components/kustomize/kustomization/#role-based-access-control | ||
| [hcsa]: https://fluxcd.io/docs/components/helm/helmreleases/#role-based-access-control | ||
| [kubeconfig]: https://fluxcd.io/docs/components/kustomize/api/#kustomize.toolkit.fluxcd.io/v1beta2.KubeConfig | ||
|
|
||
| ## Assumptions made by the multi-tenancy models | ||
|
|
@@ -167,100 +179,22 @@ The Flux CLI offers an easy way of generating all the Kubernetes manifests neede | |
| - `flux create source git` command generates the configuration that tells Flux which repositories belong to tenants. | ||
| - `flux create kustomization` command generates the configuration that tells Flux how to reconcile the manifests found in the tenants repositories. | ||
|
|
||
| All the above commands have an `--export` flag for generating the Kubernetes resources in YAML format. | ||
| The platform admins should place the generated manifests in the repository that defines the cluster(s) desired state. | ||
|
|
||
| Here is an example of the generated manifests: | ||
|
|
||
| ```yaml | ||
| --- | ||
| apiVersion: v1 | ||
| kind: Namespace | ||
| metadata: | ||
| name: tenant1 | ||
| --- | ||
| apiVersion: v1 | ||
| kind: ServiceAccount | ||
| metadata: | ||
| name: flux | ||
| namespace: tenant1 | ||
| --- | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: RoleBinding | ||
| metadata: | ||
| name: flux | ||
| namespace: tenant1 | ||
| roleRef: | ||
| apiGroup: rbac.authorization.k8s.io | ||
| kind: ClusterRole | ||
| name: cluster-admin | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: flux | ||
| namespace: tenant1 | ||
| --- | ||
| apiVersion: source.toolkit.fluxcd.io/v1beta1 | ||
| kind: GitRepository | ||
| metadata: | ||
| name: tenant1 | ||
| namespace: tenant1 | ||
| spec: | ||
| interval: 5m0s | ||
| ref: | ||
| branch: main | ||
| secretRef: | ||
| name: tenant1-git-auth | ||
| url: ssh://[email protected]/org/tenant1 | ||
| --- | ||
| apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 | ||
| kind: Kustomization | ||
| metadata: | ||
| name: tenant1 | ||
| namespace: tenant1 | ||
| spec: | ||
| interval: 10m0s | ||
| path: ./ | ||
| prune: true | ||
| serviceAccountName: flux | ||
| sourceRef: | ||
| kind: GitRepository | ||
| name: tenant1 | ||
| ``` | ||
| Note that the [cluster-admin](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) | ||
| role is used in a `RoleBinding`; this only gives full control over every resource in the role binding's namespace. | ||
|
|
||
| Once the tenants main repositories are registered on the cluster(s), the tenants can configure their app delivery | ||
| in Git using Kubernetes namespace-scoped resources such as `Deployments`, `Services`, Flagger `Canaries`, | ||
| Flux `GitRepositories`, `Kustomizations`, `HelmRepositories`, `HelmReleases`, `ImageUpdateAutomations`, | ||
| `Alerts`, `Receivers`, etc. | ||
|
|
||
| #### Caveats | ||
|
|
||
| As of v0.23.0, Flux does not enforce a service account to be specified on Flux `Kustomizations` and `HelmReleases`. | ||
| When a service account is not specified, Flux defaults to cluster-admin. | ||
| In order to enforce the tenant isolation, an admission controller such as Kyverno or OPA Gatekeeper must be used | ||
| to make the `.spec.serviceAccountName` a required field for the Flux custom resources created by tenants. | ||
|
|
||
| We provide an [example](https://github.com/fluxcd/flux2-multi-tenancy/blob/main/infrastructure/kyverno-policies/flux-multi-tenancy.yaml) | ||
| for enforcing service accounts using a Kyverno cluster policy. | ||
|
|
||
| As of v0.23.0, Flux allows for `Kustomizations` and `HelmReleases` to reference sources | ||
| (`GitRepositories`, `HelmRepositories` and `Buckets`) across namespaces. | ||
| In order to prevent tenants from accessing each other sources, an admission controller such as Kyverno or OPA Gatekeeper | ||
| must be used to block cross-namespace references. | ||
|
|
||
| We provide an [example](https://github.com/fluxcd/flux2-multi-tenancy/blob/main/infrastructure/kyverno-policies/flux-multi-tenancy.yaml) | ||
| for blocking source cross-namespace references using a Kyverno cluster policy. | ||
|
|
||
| ### Hard Multi-Tenancy | ||
|
|
||
| With hard multi-tenancy, the platform admins use Kubernetes Cluster API to create dedicated clusters for each tenant. | ||
| The Flux instance installed on the management cluster is responsible | ||
| for reconciling the cluster definitions belonging to tenants. | ||
| With hard multi-tenancy, the platform admins create dedicated clusters for each tenant. | ||
|
|
||
| When the tenants's clusters are created with Kubernetes Cluster API, the Flux instance | ||
| installed on the management cluster is responsible for reconciling the cluster | ||
| definitions belonging to tenants. | ||
|
|
||
| To enable GitOps for the tenant's clusters, the platform admins can configure the Flux instance running on the | ||
| management cluster to connect to the tenant's cluster using the `kubeConfig` generated by the Cluster API provider. | ||
| management cluster to connect to the tenant's cluster using the kubeconfig generated by the Cluster API provider | ||
| or by creating kubeconfig secrets for the clusters created by other means than Cluster API. | ||
|
|
||
| To configure Flux reconciliation of remote clusters, a Kubernetes secret containing a `kubeConfig` can be specified | ||
| in Flux `Kustomizations` and `HelmReleases` under `.spec.kubeConfig.secretRef`. Please consult the Flux API | ||
|
|
@@ -278,7 +212,7 @@ When using a Kubernetes Cluster API provider, the `kubeConfig` secret is automat | |
| make use of it without any manual actions. For clusters created by other means than Cluster API, the | ||
| platform team has to create the `kubeConfig` secrets to allow Flux access to the remote clusters. | ||
|
|
||
| As of Flux v0.23.0, we don't provide any guidance for cluster admins on how to generate the `kubeConfig` secrets. | ||
| As of Flux v0.24 (Nov 2021), we don't provide any guidance for cluster admins on how to generate the `kubeConfig` secrets. | ||
|
|
||
| ## Implementation History | ||
|
|
||
|
|
||