Support for creating service accounts with IAM policy #525

msvticket · 2019-09-24T13:30:22Z

I have issues

I'm submitting a...

bug report
feature request
support request
kudos, thank you, warm fuzzy

What is the current behavior?

Since version 6.0.0 the module supports outputting the OIDC issuer URL. With this you can go through the steps documented here to create a service account with permissions for a policy.

What's the requested behavior?

It would be nice to have this automated. Allowing to do something like this:

data "aws_iam_policy_document" "dns" {
    statement {
        sid       = "AllowRoute53Change"
        effect    = "Allow"
        actions   = [
            "route53:ChangeResourceRecordSets"
        ]
        resources = ["arn:aws:route53:::hostedzone/*"]
    }
    statement {
        sid       = "AllowRoute53List"
        effect    = "Allow"
        actions   = [
            "route53:ListHostedZones",
            "route53:ListResourceRecordSets"
        ]
        resources = ["*"]
    }
}

resource "aws_iam_policy" "dns" {
    name        = "EKSExternalDNS"
    policy      = data.aws_iam_policy_document.dns.json
}

module "eks" {
    source             = "terraform-aws-modules/eks/aws"
    ...
    iamserviceaccounts = {
       "kube-system/externaldns" = aws_iam_policy.dns.arn
       "default/s3reader"        = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
   }
}

where iamserviceaccounts would be a new map variable in the module with serviceaccounts as keys (qualified with name space before the slash) and policy ARNs as values.

The text was updated successfully, but these errors were encountered:

max-rocket-internet · 2019-09-25T15:55:25Z

Hi @msvticket

In that example, you are essentially just moving 2 resources into this module (aws_iam_role and aws_iam_policy_attachment) and we would loop over the contents of the iamserviceaccounts map and create these resources.

I get the idea but I think it makes more sense to just create your own module to do this.

Feel free to convince me otherwise 😄

rverma-nikiai · 2019-09-25T18:12:14Z

It would be nice to include a complete example in this module. Maybe we can leverage autoscaler and expose it under default serviceAccounts but different iam role than worker.

abdennour · 2019-09-25T19:48:58Z

May be a lot of people don't know the context of this feature request :

Indeed, this feature has been released since EKS k8s 1.13

Announcement

Amazon EKS Adds Support to Assign IAM Permissions to Kubernetes Service Accounts

AWS Docs

max-rocket-internet · 2019-09-26T14:48:38Z

Maybe we can leverage autoscaler and expose it under default serviceAccounts

Yes I think this would be a good idea. Include an example of creating an IAM role and policy to use for the cluster-autoscaler. And also remove the autoscaling policy from the worker group.

dcherman · 2019-09-30T20:07:15Z

Agreed on the autoscaling example with one caveat - we should hold off on that particular one until cluster-autoscaler actually supports it :)

They need to update their AWS SDK in order to do so:

kubernetes/autoscaler#2335

barryib · 2019-10-01T07:58:27Z

Agreed with @dcherman. For now, only the autoscaler's master branch has been updated, see kubernetes/autoscaler#2323 (so for Kubernetes 1.16).

For autoscaler which is compatible with Kubernetes 1.13 and 1.14 here the PR

Update golang sdk in 1.13 branch to use iam service account kubernetes/autoscaler#2356
Update golang sdk in 1.14 branch to use iam service account kubernetes/autoscaler#2358

In short, it doesn't work on EKS for now.

barryib · 2019-10-17T00:02:49Z

FYI, it sounds like it's now backported into 1.13 and 1.14 kubernetes/autoscaler#2301 (comment)

miguelaferreira · 2019-10-17T11:10:19Z

In that example, you are essentially just moving 2 resources into this module (aws_iam_role and aws_iam_policy_attachment) and we would loop over the contents of the iamserviceaccounts map and create these resources.

I get the idea but I think it makes more sense to just create your own module to do this.

For anyone interested, I've extended the terraform-aws-iam module to setup the IAM role and assume role policies.
Pending PR here: terraform-aws-modules/terraform-aws-iam#37

Edit:
In another module I wire it all together and also annotate the k8s service account: https://gitlab.com/open-source-devex/terraform-modules/aws/iam-role-for-k8s-service-account

stale · 2020-01-19T14:29:53Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

0verc1ocker · 2020-01-22T22:01:26Z

This would be really nice to have and to provide an example for how to do it with the cluster-autoscaler now. cluster-autoscaler now supports it in 1.14.6?

I am interested and could work on this one.

morganchristiansson · 2020-01-23T09:18:19Z

Some support for IRSA addied in the v8.0.0 release https://github.com/terraform-aws-modules/terraform-aws-eks/releases/tag/v8.0.0

max-rocket-internet · 2020-01-23T11:44:03Z

I'm still not convinced we want to manage IAM roles and policy in the core of this module. You could perhaps add a separate module under /modules but is it really that complicated? People can just manage IAM stuff elsewhere in their TF code base?

morganchristiansson · 2020-01-23T12:40:39Z

Yeah I have an iam.tf file next to eks.tf which works great.

A guide to enable and migrate to IRSA would be great if the support is at that stage.

max-rocket-internet · 2020-01-23T14:47:50Z

A guide to enable and migrate to IRSA would be great if the support is at that stage.

100%. We'll make an example or accept a PR for this.

TBH I'm not sure the amazon-eks-pod-identity-webhook thing is ready. We've been waiting months for official docker images and today I tried their installation approach and hit some issues aws/amazon-eks-pod-identity-webhook#33

dpiddockcmp · 2020-01-23T15:33:53Z

But the webhook is built in to EKS for 1.14 and 1.13 clusters created or upgraded after September 3rd. You don't need to run anything extra. I've gone through the painful process of migrating from kube2iam to OIDC on a live cluster.

max-rocket-internet · 2020-01-23T15:42:22Z

You don't need to run anything extra

Not even the eks-pod-identity-webhook pods? If you check the pod-identity-webhook mutatingwebhookconfiguration, it has this URL:

url: https://127.0.0.1:23443/mutate

What is serving that? The EKS service?

dpiddockcmp · 2020-01-23T15:52:51Z

It's all hidden away out of sight. I'm guessing on the master nodes?

You can spin up a new cluster and annotate a ServiceAccount with the necessary eks.amazonaws.com/role-arn stuff. New pods using that account will modified. Points 3 and 4 the project's README: https://github.com/aws/amazon-eks-pod-identity-webhook

max-rocket-internet · 2020-01-23T17:06:45Z

@dpiddockcmp

It's all hidden away out of sight. I'm guessing on the master nodes?

Thank you. It appears I skim read the documentation and the AWS announcements a missed some vital information 😅I did think it was somewhat lazy of AWS not to run this thing themselves....

Anyway, looking at creating an example for the cluster-autoscaler, and I see there's these resources required:

aws_iam_role (for the pods)
aws_iam_policy_document (assume role policy)
aws_iam_policy_document (for the pods)
aws_iam_policy
aws_iam_role_policy_attachment

And the required data for these resources:

name of the k8s ServiceAccount
oidc_provider_arn
cluster_oidc_issuer_url

What do you think about creating a module for this?

morganchristiansson · 2020-01-23T17:22:49Z

aws_iam_role (for the pods)

aws_iam_policy_document (assume role policy)

aws_iam_policy_document (for the pods)

aws_iam_policy

aws_iam_role_policy_attachment

Can be with just 2 resources:

aws_iam_role with inline assume_role_policy
aws_iam_role_policy with inline policy attached to role above

Per https://www.terraform.io/docs/providers/aws/r/iam_role_policy.html

max-rocket-internet · 2020-01-23T17:31:47Z

Can be with just 2 resources

Sure but it looks ugly 💅

0verc1ocker · 2020-01-23T19:29:54Z

Would really appreciate an example for the cluster-autoscaler as well using IAM roles for Service Accounts for EKS. I am currently working on some cluster environments in EKS and although I do have a terraform iam module for certain resources/workflows, it feels like these iam roles should be within this module as they need to be configured with the service accounts together.

Since the ServiceAccount abstraction is a Kubernetes concept, it would be really nice to have some examples in here on how to do it, especially with the OICD provider.

Spent last year working with GKE clusters, they had really good docs/guides on how to do all this with their roles and policies inherent in the GCE platform.

100%. We'll make an example or accept a PR for this.

Thank you! 🙇‍♂️

0verc1ocker · 2020-01-23T19:31:13Z

Is there any documentation on what the https://github.com/aws/amazon-eks-pod-identity-webhook actually does? Is this just like kube2iam but for EKS clusters using the OICD provider?

dpiddockcmp · 2020-01-24T09:32:13Z

The gitrepo explains what the webhook does but not really why or how you would use it.

The announcement blog post contains more details on the why and how.

There's also the eksworkshop section about IAM roles but it's a little eksctl walk-through and less on why.

I think we should create an example of how all this works in Terraform with this module.

I'm on the fence over whether creating the IAM roles, profiles, namespaces and service accounts should be done in the module. That's massive feature creep.

max-rocket-internet · 2020-01-24T13:47:26Z

I think we should create an example of how all this works in Terraform with this module.

OK done: #710

Can some people have a look and maybe test the example?

I'm on the fence over whether creating the IAM roles, profiles, namespaces and service accounts should be done in the module. That's massive feature creep.

I think I agree. Let's not add anything for this.

morganchristiansson · 2020-01-24T17:59:46Z

cluster-autoscaler or IAM roles is not currently managed by EKS so no need to start doing so. Just an example to get started is fine. I suppose some pre-created IAM roles for cluster-autoscaler could be useful tho..

I am using this simple terraform module to create IAM roles with inline policies. See main.tf for usage - this suits my needs to easily create and manage IAM roles for kube2iam currently and should work similarly for IRSA. It doesn't support policy attachments but I've ended up not using them. Tho it's easy to create variations of the module or extend it.. https://gist.github.com/morganchristiansson/9220a94b4ba5af21107031c8cd783960

morganchristiansson · 2020-01-24T19:28:43Z

It just occured to me that there might be existing terraform module for iam roles.. and yes there is.

https://github.com/terraform-aws-modules/terraform-aws-iam/blob/master/examples/iam-assumable-role/main.tf

Does what my module does and more.

max-rocket-internet · 2020-01-27T09:08:32Z

I think we should use a module from this repo: terraform-aws-modules/terraform-aws-iam#41

antonbabenko · 2020-01-27T09:30:47Z

terraform-aws-iam module v2.5.0 has been just released with IAM assumable role with OIDC.

Thanks @miguelaferreira , as usual !

max-rocket-internet · 2020-01-30T14:24:36Z

OK I'm gonna close this now. An example has been added and also a module you can use.

github-actions · 2022-11-28T02:19:25Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

stale bot added the stale label Jan 19, 2020

stale bot removed the stale label Jan 22, 2020

max-rocket-internet closed this as completed Jan 30, 2020

github-actions bot locked as resolved and limited conversation to collaborators Nov 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for creating service accounts with IAM policy #525

Support for creating service accounts with IAM policy #525

msvticket commented Sep 24, 2019

max-rocket-internet commented Sep 25, 2019

rverma-nikiai commented Sep 25, 2019

abdennour commented Sep 25, 2019 •

edited

Loading

max-rocket-internet commented Sep 26, 2019

dcherman commented Sep 30, 2019

barryib commented Oct 1, 2019 •

edited

Loading

barryib commented Oct 17, 2019

miguelaferreira commented Oct 17, 2019 •

edited

Loading

stale bot commented Jan 19, 2020

0verc1ocker commented Jan 22, 2020 •

edited

Loading

morganchristiansson commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020

morganchristiansson commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020

dpiddockcmp commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020

dpiddockcmp commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020 •

edited

Loading

morganchristiansson commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020

0verc1ocker commented Jan 23, 2020

0verc1ocker commented Jan 23, 2020

dpiddockcmp commented Jan 24, 2020

max-rocket-internet commented Jan 24, 2020

morganchristiansson commented Jan 24, 2020

morganchristiansson commented Jan 24, 2020

max-rocket-internet commented Jan 27, 2020

antonbabenko commented Jan 27, 2020

max-rocket-internet commented Jan 30, 2020

github-actions bot commented Nov 28, 2022

Support for creating service accounts with IAM policy #525

Support for creating service accounts with IAM policy #525

Comments

msvticket commented Sep 24, 2019

I have issues

I'm submitting a...

What is the current behavior?

What's the requested behavior?

max-rocket-internet commented Sep 25, 2019

rverma-nikiai commented Sep 25, 2019

abdennour commented Sep 25, 2019 • edited Loading

Announcement

AWS Docs

max-rocket-internet commented Sep 26, 2019

dcherman commented Sep 30, 2019

barryib commented Oct 1, 2019 • edited Loading

barryib commented Oct 17, 2019

miguelaferreira commented Oct 17, 2019 • edited Loading

stale bot commented Jan 19, 2020

0verc1ocker commented Jan 22, 2020 • edited Loading

morganchristiansson commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020

morganchristiansson commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020

dpiddockcmp commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020

dpiddockcmp commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020 • edited Loading

morganchristiansson commented Jan 23, 2020

max-rocket-internet commented Jan 23, 2020

0verc1ocker commented Jan 23, 2020

0verc1ocker commented Jan 23, 2020

dpiddockcmp commented Jan 24, 2020

max-rocket-internet commented Jan 24, 2020

morganchristiansson commented Jan 24, 2020

morganchristiansson commented Jan 24, 2020

max-rocket-internet commented Jan 27, 2020

antonbabenko commented Jan 27, 2020

max-rocket-internet commented Jan 30, 2020

github-actions bot commented Nov 28, 2022

abdennour commented Sep 25, 2019 •

edited

Loading

barryib commented Oct 1, 2019 •

edited

Loading

miguelaferreira commented Oct 17, 2019 •

edited

Loading

0verc1ocker commented Jan 22, 2020 •

edited

Loading

max-rocket-internet commented Jan 23, 2020 •

edited

Loading