How to set the minsize and maxsize of the ec2 in karpenter provisioner?

[yzhengwei]

Version
Karpenter: v0.5.3

Kubernetes: v1.2.0

How to set the minsize and maxsize of the ec2 in karpenter provisioner? Fox example, When I have 10 nodes, how do I manually adjust to 100? And when scaled downs, how to set the parallelism and idle time ?

[suket22]

I'm not sure I follow your ask.

How to set the minsize and maxsize of the ec2 in karpenter provisioner?

Do you mean the minimum and maximum bounds on the number of EC2 instances Karpenter will provision, or the actual sizes of the instances Karpenter is choosing? If it's the former, you can specify an upper bound by using spec.Limits.Resources in terms of CPU and Memory (not the actual number of nodes as you would if using ASG).

Karpenter scales to zero which is the default min bound. We don't today support any other min bound, since that generally results in unnecessary costs. We are looking to drive down the time it takes for a node to launch in response to a pending pod (~50s currently) so a min bound of 0 should fit most use cases.

And when scaled downs, how to set the parallelism and idle time ?

What do you mean by idle time? Karpenter will attach a label signifying a TTL on any empty node, and cordon drain it 30 seconds later. This process happens for each worker node individually.

[balonik]

Hi,
minimum number of resources available is really a must-have feature even if it is not cost-efficient.
I would say for most solutions you need to scale quicker from 0 to catch the initial load and (maybe) than you can scale slower, but waiting 50s for a first pod to be scheduled is too long.
My use-case would be having a "shared" provisioner for 2 solutions which need to react fast for spikes in workload. But I don't want to have idle pods sitting around and taking up compute resources. I need to have a minimum set of nodes ready so either of the solution can quickly scale up on the available nodes. Something like compute resources buffer.
It could work that karpenter would always provision nodes for real demand + buffer.

[winterrobert]

Yeah are also looking for something like this even though I guess it's a bit out of scope for the Karpenter project.

We like the Karpenter features but it would also be great to be able to set a number and type of base instances that are always provisioned even if empty.

[ellistarn]

One option is to use an EKS managed node group or ASG to manage this static capacity. Can you elaborate on the value you see that karpenter could provide in this space? Is it simply that you want a single system that behaves similarly for both use cases? FWIW, I'm totally open minded to including something that accomplishes this in Karpenter.

[ellistarn]

Closing in favor of kubernetes-sigs/karpenter#749