diff --git a/cluster-autoscaler/cloudprovider/aws/README.md b/cluster-autoscaler/cloudprovider/aws/README.md index 36f5d7a2849..fb2f33ce30b 100644 --- a/cluster-autoscaler/cloudprovider/aws/README.md +++ b/cluster-autoscaler/cloudprovider/aws/README.md @@ -1,5 +1,5 @@ # Cluster Autoscaler on AWS -The cluster autoscaler on AWS scales worker nodes within any specified autoscaling group. It will run as a `Deployment` in your cluster. This README will go over some of the necessary steps required to get the cluster autoscaler up and running. +The cluster autoscaler on AWS scales worker nodes within any specified autoscaling group. It will run as a `Deployment` in your cluster. This README covers the steps required to configure and run the cluster autoscaler. ## Kubernetes Version Cluster autoscaler must run on v1.3.0 or greater. @@ -27,35 +27,23 @@ A minimum IAM policy would look like: ] } ``` +If you'd like to scale node groups from 0, an +`autoscaling:DescribeLaunchConfigurations` or +`ec2:DescribeLaunchTemplateVersions` permission is required depending on if you +made your ASG with Launch Configuration or Launch Template. -If you'd like to auto-discover node groups by specifying the `--node-group-auto-discovery` flag, a `DescribeTags` permission is also required: +If you'd like the cluster autoscaler to [automatically +discover](#auto-discovery-setup) EC2 AutoScalingGroups, the +`autoscaling:DescribeTags` permission is also required. -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "autoscaling:DescribeAutoScalingGroups", - "autoscaling:DescribeAutoScalingInstances", - "autoscaling:DescribeLaunchConfigurations", - "autoscaling:DescribeTags", - "autoscaling:SetDesiredCapacity", - "autoscaling:TerminateInstanceInAutoScalingGroup" - ], - "Resource": "*" - } - ] -} -``` - -AWS supports ARNs for autoscaling groups. More information [here](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources). +**NOTE**: You can restrict the target resources for the autoscaling actions by +specifying autoscaling group ARNS. More information can be found +[here](https://docs.aws.amazon.com/autoscaling/latest/userguide/control-access-using-iam.html#policy-auto-scaling-resources). ### Using AWS Credentials -For on premise users like to scale out to AWS, above approach that attaching policy to nodegroup role won't work. Instead, you can create an aws secret manually and add following environment variables to cluster-autoscaler deployment manifest. Cluster autoscaler will use credential to authenticate and authorize itself. Please make sure your role has above permissions. +For on premise users wishing to scale out to AWS, the above approach of attaching policy to a nodegroup role won't work. Instead, you can create an aws secret manually and add following environment variables to cluster-autoscaler deployment manifest. Cluster autoscaler will use credential to authenticate and authorize itself. Please make sure your role has above permissions. -``` +```yaml apiVersion: v1 kind: Secret metadata: @@ -64,11 +52,10 @@ type: Opaque data: aws_access_key_id: BASE64_OF_YOUR_AWS_ACCESS_KEY_ID aws_secret_access_key: BASE64_OF_YOUR_AWS_SECRET_ACCESS_KEY - ``` Please check [guidance](https://kubernetes.io/docs/concepts/configuration/secret/#creating-a-secret-manually) for creating a secret manually. -``` +```yaml env: - name: AWS_ACCESS_KEY_ID valueFrom: @@ -82,7 +69,6 @@ env: key: aws_secret_access_key - name: AWS_REGION value: YOUR_AWS_REGION - ``` ## Deployment Specification @@ -108,7 +94,6 @@ Please replace `{{ node_asg_min }}`, `{{ node_asg_max }}` and `{{ name }}` with kubectl apply -f examples/cluster-autoscaler-run-on-master.yaml ``` - ### Auto-Discovery Setup To run a cluster-autoscaler which auto-discovers ASGs with nodes use the `--node-group-auto-discovery` flag. For example, `--node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/` will find the ASGs where those tag keys @@ -204,13 +189,6 @@ spec: - r5ad.2xlarge ``` -## Use Static Instance List -The set of the latest supported EC2 instance types will be fetched by the CA at run time. You can find all the available instance types in the CA logs. -If your network access is restricted such that fetching this set is infeasible, you can specify the command-line flag `--aws-use-static-instance-list=true` to switch the CA back to its original use of a statically defined set. - -To refresh static list, please run `go run ec2_instance_types/gen.go` under `cluster-autoscaler/cloudprovider/aws/` and update `staticListLastUpdateTime` in `aws_util.go` - - ### Example usage: * Create a [Launch Template](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-autoscaling-autoscalinggroup-launchtemplate.html) (LT) with an instance type, for example, r5.2xlarge. Consider this the 'base' instance type. Do not define any spot purchase options here. @@ -222,6 +200,12 @@ To refresh static list, please run `go run ec2_instance_types/gen.go` under `clu See CloudFormation example [here](MixedInstancePolicy.md). +## Use Static Instance List +The set of the latest supported EC2 instance types will be fetched by the CA at run time. You can find all the available instance types in the CA logs. +If your network access is restricted such that fetching this set is infeasible, you can specify the command-line flag `--aws-use-static-instance-list=true` to switch the CA back to its original use of a statically defined set. + +To refresh static list, please run `go run ec2_instance_types/gen.go` under `cluster-autoscaler/cloudprovider/aws/` and update `staticListLastUpdateTime` in `aws_util.go` + ## Common Notes and Gotchas: - The `/etc/ssl/certs/ca-bundle.crt` should exist by default on ec2 instance in your EKS cluster. If you use other cluster privision tools like [kops](https://github.com/kubernetes/kops) with different operating systems other than Amazon Linux 2, please use `/etc/ssl/certs/ca-certificates.crt` or correct path on your host instead for the volume hostPath in your cluster autoscaler manifest. - Cluster autoscaler does not support Auto Scaling Groups which span multiple Availability Zones; instead you should use an Auto Scaling Group for each Availability Zone and enable the [--balance-similar-node-groups](../../FAQ.md#im-running-cluster-with-nodes-in-multiple-zones-for-ha-purposes-is-that-supported-by-cluster-autoscaler) feature. If you do use a single Auto Scaling Group that spans multiple Availability Zones you will find that AWS unexpectedly terminates nodes without them being drained because of the [rebalancing feature](https://docs.aws.amazon.com/autoscaling/ec2/userguide/auto-scaling-benefits.html#arch-AutoScalingMultiAZ).