[EKS] Support for swap memory with Kubernetes

[asortino]

Tell us about your request
EKS alpha support for using swap memory

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
In Kubernetes v1.22, alpha support for using swap memory was introduced. However, on EKS changelog for Kubernetes v1.22 this feature does not appear.

[agalligari]

It would be great at least for EC2 managed node groups

[Pablo-platypus]

any updates on this?

[dims]

It is still alpha as of today (Kubernetes folks are working on 1.27 and it's not on the table for 1.27 either)
https://github.com/kubernetes/kubernetes/blob/090025f5e6525b77017c6874b96e2569a6a597c4/pkg/features/kube_features.go#L566-L570

[wm-nimesh]

Moved to beta in 1.28 -> https://kubernetes.io/blog/2023/08/15/kubernetes-v1-28-release/#beta-support-for-enabling-swap-space-on-linux

[OrBin]

Any update on this? Is there any workaround besides just waiting until it's stable?

[dims]

@OrBin see requirements for this feature, it needs cgroups v2 on your node ( https://kubernetes.io/blog/2023/08/24/swap-linux-beta/#how-do-i-use-it ) our default AMI(s) do not have cgroups v2 yet. So you would need a custom AMI with an operating system with cgroups v2 with a custom kublet configuration as mentioned in the blog. There is no configuration on the apiserver side to make it work with 1.28 i think. So if you really want to try you will have to do what i mentioned before. When we get to AL2023, things like this will be easier to enable by default.

[OrBin]

@OrBin see requirements for this feature, it needs cgroups v2 on your node ( kubernetes.io/blog/2023/08/24/swap-linux-beta/#how-do-i-use-it ) our default AMI(s) do not have cgroups v2 yet. So you would need a custom AMI with an operating system with cgroups v2 with a custom kublet configuration as mentioned in the blog. There is no configuration on the apiserver side to make it work with 1.28 i think. So if you really want to try you will have to do what i mentioned before. When we get to AL2023, things like this will be easier to enable by default.

Your answer applies for Kubernetes 1.28, right? I am actually using an older version, so I don't think the cgroup version is an issue for me.

[dims]

@OrBin the feature is NOT available in any other version of k8s by default.

[goloveychuk]

I see that eks now uses al2023, which have cgroups2. What is left to support swap?

[dims]

please watch kubernetes/enhancements#2400

[LouAdrienCEL]

+1

[cartermckinnon]

This is supported on the AL2023-based EKS AMI's, you'll just need to pass the necessary options to kubelet in your NodeConfig. We do not currently plan to enable this feature by default.

This feature will not be supported on AL2-based EKS AMI's, as cgroupv2 is not supported there.

[uyilmaz]

I guess it graduated to stable with k8s 1.31? Not the most knowledgeable person on this though

[dims]

It's beta since 1.30 - https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2400-node-swap/kep.yaml#L33-L36

You can read all about it here - https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2400-node-swap/README.md

Like @cartermckinnon pointed out, the feature will ONLY work in Operating Systems that use cgroups v2, So Amazon Linux 2023 qualifies and AL2 does NOT:
https://docs.aws.amazon.com/linux/al2023/ug/cgroupv2.html

Just to be clear, It does not matter if it is graduated or not, it will NOT work on cgroups v1 (AL2).

Hope this helps!