EKS - Kubernetes 1.11 Upgrade

[mmcaya]

I have issues

With the release of EKS 1.11 Upgrades, this module should decide it default posture on upgrades moving forward.

https://aws.amazon.com/blogs/compute/making-cluster-updates-easy-with-amazon-eks/

Questions for maintainers @max-rocket-internet @brandoconnor

Cluster Defaults
Should this module always default to the current 1.10 setup, and require users to explicitly upgrade (favors existing users of this module)
OR
Should this module adhere to the current EKS default (e.g. 1.11.5), requiring existing users to explicitly set the version (e.g. 1.10) if using the previous default and they don't wish to upgrade their cluster on the next terraform module release (e.g. 1.9.0)

I personally lean towards matching the EKS defaults (which is what the terraform aws provider does), and proper communication in CHANGELOG, and encouraging module users to explicitly set their version rather than depend on the default value.

Note, to confirm, changing the version currently forces a new resource at this time
version: "1.10" => "1.11" (forces new resource)

Default AMI Filter
Current AMI filter is now too generic for the new EKS optimized AMI naming convention:

Prior to 1.11 upgrade availability: amazon-eks-node-vXX
After: amazon-eks-node-<K8S-VERSION>-v<DATE>

The only reason the current module will continue to pick the 1.10 AMI to match its current 1.10 cluster default is because AWS provisioned it later than the 1.11 AMI

1.11.X = amazon-eks-node-1.11-v20181210 = ami-0b4eb1d8782fc3aea = December 6, 2018 at 10:29:16 PM UTC-5
1.10.X = amazon-eks-node-1.10-v20181210 = ami-027792c3cc6de7b5b = December 6, 2018 at 10:50:25 PM UTC-5

If AWS creates a new 1.11 AMI after a new 1.10 AMI, this module will pick that up the 1.11 one as latest, potentially mis-matching with the defaults.

I'm submitting a...

• bug report
• feature request
• support request
• kudos, thank you, warm fuzzy

What's the expected behavior?

Upgrade posture documented and AMI filters updated to match upgrade posture.

Are you able to fix this problem and submit a PR? Link here if you have already.

I can depending on the decision by module maintainers on upgrade support.

[max-rocket-internet]

Current AMI filter is now too generic for the new EKS optimized AMI naming convention:

Agree 100%. In my experience the current behaviour automatically updates the AMI, which is not good.

I think we need to wait for the terraform resource aws_eks_cluster to support upgrades, right?

[mmcaya]

I was just investigating that side of things. Yeah, any actual changes should be in line with the provider hopefully not forcing a new resource 🤞 I can always split the AMI filter issue from the upgrade issue so that the module at least is consistently using 1.10 resources by default prior to upgrade support.

[max-rocket-internet]

I can always split the AMI filter issue from the upgrade issue

That would be great, it needs fixing anyway 💛

[mmcaya]

Might not have to wait too long for either.
Support for upgrading EKS Clusters via the Terraform AWS provider has been merged and will release later today with version 1.52.0.

[mmcaya]

For AMI versioning, are you thinking pinning it to a specific AMI revision per module release? e.g. amazon-eks-node-${var.cluster_version}-v20181210 would be the default AMI for 1.9.0 making it always deterministic (per cluster version)?

[max-rocket-internet]

Something like that. Or can we select the latest minor version of the major cluster version using the AMI data source filters?

[max-rocket-internet]

Regarding AMI issue, how does this look? #215

[max-rocket-internet]

In regards to upgrading existing clusters, I just triggered the update on the EKS console, it took about 25 mins. Then updated to the latest v2.0.0 version of this module and then everything is fine. No TF changes. Not an ideal process but works fine.

[qgriffith-xylem]

@max-rocket-internet does that mean you left your value for the cluster version to 1.10 in the tf file after the upgrade you did via the console?

[mmcaya]

Went through the upgrade process via terraform directly with 2.0.0 tag - no UI

• Default values for version 1.11 - Good
• Have a blue / green worker node setup, so pinned the old nodes to a specific AMI, and then used the default AMI filter for the new 1.11 nodes - Good
• Took about 25 min as well

[alexsn]

Is it possible to update existing workers to new AMI?

[max-rocket-internet]

does that mean you left your value for the cluster version to 1.10 in the tf file

Nope because updating to version v2.0.0 of this module means the value is then 1.11.

Went through the upgrade process via terraform directly with 2.0.0 tag

So TF did the upgrade? For me it wanted to recreate the whole cluster?

[mmcaya]

Yeah, as long as you have the updated aws provider (1.52.0), it works as expected.

Found version 2.0.0 of terraform-aws-modules/eks/aws on registry.terraform.io
...
- Downloading plugin for provider "aws" (1.52.0)...
...
module.eks.aws_eks_cluster.this: Modifying... (ID: XXX)
  version: "1.10" => "1.11"
...
module.eks.aws_eks_cluster.this: Still modifying... (ID: XXX, 19m9s elapsed)
module.eks.aws_eks_cluster.this: Modifications complete after 19m19s (ID: XXX)

[max-rocket-internet]

I think we can close this now.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.