From bffaef5e6a61b040bb4b6482953c81f3d28a3a60 Mon Sep 17 00:00:00 2001 From: GuyTempleton Date: Tue, 8 Feb 2022 22:45:00 +0000 Subject: [PATCH 1/2] CA - AWS - Rework IAM Docs/Snippets --- .../cloudprovider/aws/README.md | 77 +++++++++++++++---- 1 file changed, 63 insertions(+), 14 deletions(-) diff --git a/cluster-autoscaler/cloudprovider/aws/README.md b/cluster-autoscaler/cloudprovider/aws/README.md index b74795f97349..c4064ad7e931 100644 --- a/cluster-autoscaler/cloudprovider/aws/README.md +++ b/cluster-autoscaler/cloudprovider/aws/README.md @@ -19,7 +19,23 @@ the EC2 instance on which the Cluster Autoscaler pod runs. ### IAM Policy -The following policy provides the minimum privileges necessary for Cluster Autoscaler to run: +There are a number of ways to run the autoscaler in AWS, which can significantly +impact the range of IAM permissions required for the Cluster Autoscaler to function +properly. Two options are provided below, one which will allow use of all of the +features of the Cluster Autoscaler, the second with a more limited range of IAM +actions enabled, which enforces using certain configuration options in the +Cluster Autoscaler binary. + +It is strongly recommended to restrict the target resources for the autoscaling actions +by either [specifying Auto Scaling Group ARNs](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources) in the `Resource` list of the policy or +[using tag based conditionals](https://docs.aws.amazon.com/autoscaling/ec2/userguide/control-access-using-iam.html#security_iam_service-with-iam-tags). The [minimal policy](#minimal-iam-permissions-policy) +includes an example of restricting by ASG ARN. + +#### Full Cluster Autoscaler Features Policy (Recommended) + +Permissions required when using [ASG Autodiscovery](#Auto-discovery-setup) and +Dynamic EC2 List Generation (the default behaviour). In this example, only the second block of actions +should be updated to restrict the resources/add conditionals: ```json { @@ -31,9 +47,17 @@ The following policy provides the minimum privileges necessary for Cluster Autos "autoscaling:DescribeAutoScalingGroups", "autoscaling:DescribeAutoScalingInstances", "autoscaling:DescribeLaunchConfigurations", + "autoscaling:DescribeTags", + "ec2:DescribeInstanceTypes", + "ec2:DescribeLaunchTemplateVersions" + ], + "Resource": ["*"] + }, + { + "Effect": "Allow", + "Action": [ "autoscaling:SetDesiredCapacity", - "autoscaling:TerminateInstanceInAutoScalingGroup", - "ec2:DescribeInstanceTypes" + "autoscaling:TerminateInstanceInAutoScalingGroup" ], "Resource": ["*"] } @@ -41,17 +65,42 @@ The following policy provides the minimum privileges necessary for Cluster Autos } ``` -If you'd like Cluster Autoscaler to [automatically -discover](#auto-discovery-setup) EC2 Auto Scaling Groups **(recommended)**, add -`autoscaling:DescribeTags` to the `Action` list. Also add -`autoscaling:DescribeLaunchConfigurations` (if you created your ASG using a -Launch Configuration) and/or `ec2:DescribeLaunchTemplateVersions` (if you -created your ASG using a Launch Template) to the `Action` list. - -If you prefer, you can restrict the target resources for the autoscaling actions -by specifying Auto Scaling Group ARNs in the `Resource` list of the policy. More -information can be found -[here](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources). +#### Minimal IAM Permissions Policy + +*NOTE:* The below policies/arguments to the Cluster Autoscaler need to be modified as appropriate +for the names of your ASGs, as well as account ID and AWS region before being used. + +The following policy provides the minimum privileges necessary for Cluster Autoscaler to run. +When using this policy, you cannot use autodiscovery of ASGs. In addition, it restricts the +IAM permissions to the node groups the Cluster Autoscaler is configured to scale. + +This in turn means that you must pass the following arguments to the Cluster Autoscaler +binary, replacing min and max node counts and the ASG: + +```bash +--aws-use-static-instance-list=false +--nodes=1:100:exampleASG1 +--nodes=1:100:exampleASG2 +``` + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "autoscaling:DescribeAutoScalingGroups", + "autoscaling:DescribeAutoScalingInstances", + "autoscaling:DescribeLaunchConfigurations", + "autoscaling:SetDesiredCapacity", + "autoscaling:TerminateInstanceInAutoScalingGroup" + ], + "Resource": ["arn:aws:autoscaling:${YOUR_CLUSTER_AWS_REGION}:${YOUR_AWS_ACCOUNT_ID}:autoScalingGroup:*:autoScalingGroupName/${YOUR_ASG_NAME}"] + } + ] +} +``` ### Using OIDC Federated Authentication From 321298a9c81f0aef8f00ae1e66e8baaa184381ec Mon Sep 17 00:00:00 2001 From: GuyTempleton Date: Wed, 16 Feb 2022 22:10:18 +0000 Subject: [PATCH 2/2] CA - AWS - Update OIDC example docs --- .../cloudprovider/aws/CA_with_AWS_IAM_OIDC.md | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/cluster-autoscaler/cloudprovider/aws/CA_with_AWS_IAM_OIDC.md b/cluster-autoscaler/cloudprovider/aws/CA_with_AWS_IAM_OIDC.md index 217bd3d505eb..cf251787c234 100644 --- a/cluster-autoscaler/cloudprovider/aws/CA_with_AWS_IAM_OIDC.md +++ b/cluster-autoscaler/cloudprovider/aws/CA_with_AWS_IAM_OIDC.md @@ -53,9 +53,11 @@ D) Set up [Cluster Autoscaler Auto-Discovery] using the [tutorial] . Note: The keys for the tags that you entered don't have values. Cluster Autoscaler ignores any value set for the keys. -- Create an IAM Policy for cluster autoscaler and to enable AutoDiscovery. +__NOTE:__ Please see [the README](README.md#IAM-Policy) for more information on best practices with this IAM role. -```sh +- Create an IAM Policy for cluster autoscaler and to enable AutoDiscovery as well as discovery of instance types. + +```json { "Version": "2012-10-17", "Statement": [ @@ -66,17 +68,23 @@ Note: The keys for the tags that you entered don't have values. Cluster Autoscal "autoscaling:DescribeAutoScalingInstances", "autoscaling:DescribeLaunchConfigurations", "autoscaling:DescribeTags", + "ec2:DescribeInstanceTypes", + "ec2:DescribeLaunchTemplateVersions" + ], + "Resource": ["*"] + }, + { + "Effect": "Allow", + "Action": [ "autoscaling:SetDesiredCapacity", "autoscaling:TerminateInstanceInAutoScalingGroup" ], - "Resource": "*" + "Resource": ["*"] } ] } ``` -NOTE: ``` autoscaling:DescribeTags ``` is very important if you are making use of the AutoDiscovery feature of the Cluster AutoScaler. - - Attach the above created policy to the *instance role* that's attached to your Amazon EKS worker nodes. - Download a deployment example file provided by the Cluster Autoscaler project on GitHub, run the following command: