Change EKS module to conform to EKS Blueprints direction

[ntwkninja]

https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/docs/v5-direction/DIRECTION_v5.md#notable-changes

[RothAndrew]

More info needed. @zack-is-cool are you working on stuff for this already? I remember you had mentioned it. Need better context on what is needed and what is already being worked.

[zack-is-cool]

Currently working it in branch: https://github.com/defenseunicorns/iac/tree/calico_cni which will be a refactor from using aws-blueprints for the EKS cluster + worker node/kms deployments to using the upstream terraform-aws-eks module. One of the main reasons for this refactor is that the upstream terraform-aws-eks module has an extra switch for deploying and configuring an EKS addons before the worker nodes are deployed, specifically configuring vpc-cni on the cluster before nodes are deployed. A benefit from this is that the pods can be configured to utilize a different subnet than the worker nodes, this will deal with ip exhaustion issues that have come from downstream projects.

We'll also be using calico to utilize network policies.
relevant links:
aws calico
calico on aws eks

@ntwkninja anything I'm missing here?

[RothAndrew]

@zack-is-cool would you mind pushing up a draft PR?

Will this work that you are doing also cover #75?

[ntwkninja]

spot on @zack-is-cool

VPC CNI with custom networking + calico CNI could not be deployed with EKS blueprints because it was missing the before_compute flag and EKS blueprints has pivoted so we couldn't work an upstream PR

@RothAndrew Zack is working #73 & #75

[RothAndrew]

Sounds good. Recommend we get the automated testing stuff in place before a big change like this gets merged.