Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autoscaler not scaling from 0 to 1 in AWS #4998

Closed
alejoGT1202 opened this issue Jun 29, 2022 · 2 comments
Closed

Autoscaler not scaling from 0 to 1 in AWS #4998

alejoGT1202 opened this issue Jun 29, 2022 · 2 comments

Comments

@alejoGT1202
Copy link

Hello,

I've a cluster with 3 different node groups. 2 of the node groups the desired and min. capacity set to 0, each node group has also an specific taint. However when I create a job that requires a node from any of this node instances the autoscaler never scales up. What I notice is that in the other node group where the desired size is set to 1, the autoscaler works as expected. I also tried to replicate the steps indicated here but it never scale up. I would like to know what steps should I follow for making the autoscaler works as expected.

Thanks for the help.

@dev-rowbot
Copy link

@alejoGT1202 - I have the same experience as you and managed to figure out the issue. If you look a the cluster autoscaler logs you will probably see it scanning all the ASG's an not finding an ASG that matches the taint. This is because eksctl will add the taint to the managed node groups in EKS but not on the EC2 Autoscaling Groups.

Open the AWS Console

  • Browse to EC2
  • Select Autoscaling Groups
  • Select your Autoscalig Group and hit edit
  • Add the missing Tags that you added as per the eksctl example

I am not sure if this is a bug with eksctl or with the cluster-autoscaler. eksctl does not add them to the ASG but cluster-autoscaler is not checking EKS for the taints. The point is that the tags do exist somewhere...

This should help you scale from 0

@alejoGT1202
Copy link
Author

Thanks for your help @dev-rowbot, I had to replace the taints for labels and then add the labels manually to my Alutoscaling group in order to make it work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants