Allow the ability to run a stack with a custom service account/role

[viveklak]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

Currently the operator service account is the same account used to run the relevant stacks. It would be ideal to allow each stack to specify the service account it should use.
e.g.:

apiVersion: pulumi.com/v1
kind: Stack
metadata:
  name: nginx-k8s-stack
spec:
...
  stack: "vivek/example/dev"
  projectRepo: https://github.com/viveklak/example
  branch: "refs/heads/master"
  destroyOnFinalize: true
  serviceAccount: <sa> # or
  role: <role>

This will probably require running the operator with a clusterrole but then "dropping privileges" to the appropriate serviceAccount/role. We may also need to consider a means to limit which roles to allow the operator to use - i.e. this is the logical equivalent of assume role in IAM in some ways and has similar security considerations.

Affected area/feature

[ghostsquad]

I think this violates a potential security principle. Segregation of actions from permissions. If the stack can define it's own permissions, (via a service account, and related RBAC or IAM role assigned to the service account), that can lead easily to privilege escalation.

What specific problem are you trying to solve though, that brought you to filing this issue?

[rajendragosavi]

I implemented the similar requirement as follows -

1. Run the kubernetes operator pod with a base service account ( I am running on EKS )
2. Pass the IAM Role ARN as config to the stack. The code running inside the stack will take that IAM Role ARN as argument and use pulumi SDK assumeRole function to do requried task.
3. that way pulumi stack can be used in a self service fashioned.

[ghostsquad]

@rajendragosavi just be aware that the self-service in this way allows anyone to gain any & all access that the operator has, even if they shouldn't have that access. A better option is to run a multiple operators, a matrix of project & environment. This ensures that an operator deployed to "development" for projectA cannot change anything in production (even accidentally), nor can it access resources from projectB in any environment.

[squaremo]

Flux went through many discussions about how to support multi-tenancy (fluxcd/flux2#2093 is as good an entry point as any ..). There's one main lesson, and one main problem.

Lesson: if you let people specify a service account or user, the default (when not supplied) should be an unprivileged account. Otherwise, just leaving out the service account name gives you the operator's permissions.

Problem: many platforms implicitly grant permissions to workloads (i.e., the operator pod), and it can be difficult to arrange different permissions when acting on behalf of a user. The most pertinent example for Pulumi is platform secret/key management -- see fluxcd/kustomize-controller#324 for an example of the problem. At best, each service or platform needs individual, careful treatment.

[EronWright]

Good news everyone, we just released a preview of Pulumi Kubernetes Operator v2. This new release has a whole-new architecture that uses pods as the execution environment. The Stack spec has a new serviceAccountName field to specify the pod's service account. Since the Pulumi program runs in the pod, it no longer uses the operator's own service account. One is expected to create a service account for the stack and to assign permissions as necessary.

Please read the announcement blog post for more information:
https://www.pulumi.com/blog/pulumi-kubernetes-operator-2-0/

Would love to hear your feedback! Feel free to engage with us on the #kubernetes channel of the Pulumi Slack workspace.