ServiceAccount can be assigned as tenant owner, but cannot be used as not in capsuleUserGroups

[slushysnowman]

Bug description

Using helm chart, v0.1.2

When assigning a ServiceAccount as tenant owner as described in the use cases, using the service account to create namespaces fails:
Error from server (Forbidden): namespaces is forbidden: User "system:serviceaccount:default:foo" cannot create resource "namespaces" in API group "" at the cluster scope

This seems to be because the service account is not in the groups defined in capsuleUserGroups.

To get this to work, system:serviceaccounts needs to be added to capsuleUserGroups, but as far as I can see this is not documented anywhere.

How to reproduce

Steps to reproduce the behavior:

1. Create a service account and assign it as owner of tenant:

kubectl create serviceaccount foo

kubectl apply -f - << EOF
apiVersion: capsule.clastix.io/v1beta1
kind: Tenant
metadata:
  name: oil
spec:
  owners:
  - name: system:serviceaccount:default:foo
    kind: ServiceAccount
EOF

2. Check auth of service account

kubectl --as system:serviceaccount:default:foo auth can-i create namespaces

Answer should be No

Expected behavior

I'd expect either:

• system:serviceaccounts to be added to capsuleUserGroups by default / in addition to any defined groups
• The requirements for the use of ServiceAccount as tenant owner to be documented, specifying that the system:serviceaccounts group needs to be added to capsuleUserGroups for Service Accounts to be able to act as tenant owners

Additional context

• Capsule version: 0.1.0
• Helm Chart version: 0.1.2
• Kubernetes version: v1.21.2-eks-0389ca3

[bsctl]

@slushysnowman thanks for reporting this.

Actually, you have to add the service account to capsule group

kubectl --as system:serviceaccount:default:robot --as-group capsule.clastix.io auth can-i create namespaces
yes

[slushysnowman]

@bsctl Yeah that's what I explored initially (especially as we aren't using the default capsule group as user group). I'm guessing it's just due to my lack of knowledge, but I couldn't find a way to modify the service account groups at all.

[arminc]

Just to verify we are saying the same thing, when capsuleUserGroups is set to a group (id) to limit the users or just to use something organization specific instead of capsule.clastix.io then user access works fine.

If a SA is created running and then run with the auth can-i create namespaces command without the --as-group from the organization it does not work. Running the command with the --as-group does work but the SA isn't going to be used that way.
It is either going to be used from a Pod (to create namespace, just an example not really) or it is going to be used from the CI/CD pipeline as credentials to access the Kubernetes and create namespaces but then it is not possible to specify the group. Unless I am missing something here?

But when the system:serviceaccounts is added to capsuleUserGroups next to the organization group id then the above described use cases do work.

@bsctl Is this how you see it as well or?

[MaxFedotov]

@arminc @slushysnowman you are both right, if you want a service account to be owner of the tenant, it should be added to userGroups in CapsuleConfiguration CRD.

We are using this feature the following way - we have a centralized argocd instance, installed in our management cluster. This argocd instance can provision applications in all our clusters. So its service account and rolebindings are created in all our clusters in argocd-system namespace. By default, when we provision a new tenant we add this account as an additional owner

  owners:
  - kind: ServiceAccount
    name: system:serviceaccount:argocd-system:argocd-manager
    proxySettings:
    - kind: IngressClasses
      operations:
      - List
    - kind: Nodes
      operations:
      - List
    - kind: StorageClasses
      operations:
      - List

And our CapsuleConfiguration spec looks like like

  userGroups:
  - kubernetes-abilitytologin
  - system:serviceaccounts:argocd-system

Because by default, each service account is a member of following groups:

system:serviceaccounts
system:serviceaccounts:{service-account-namespace}
system:authenticated

[prometherion]

I guess we should document better how to let Capsule handling the provisioner procedure for Service Account: can you take this, @bsctl?