diff --git a/examples/complete/README.md b/examples/complete/README.md index c4193a3d..5adafd93 100644 --- a/examples/complete/README.md +++ b/examples/complete/README.md @@ -92,7 +92,7 @@ kubectl get nodes | Name | Version | |------|---------| | [terraform](#requirement\_terraform) | >= 1.0.0 | -| [aws](#requirement\_aws) | >= 4.47.0 | +| [aws](#requirement\_aws) | >= 4.62.0 | | [cloudinit](#requirement\_cloudinit) | >= 2.0.0 | | [helm](#requirement\_helm) | >= 2.5.1 | | [http](#requirement\_http) | 2.4.1 | @@ -108,7 +108,7 @@ kubectl get nodes | Name | Version | |------|---------| -| [aws](#provider\_aws) | >= 4.47.0 | +| [aws](#provider\_aws) | >= 4.62.0 | | [random](#provider\_random) | >= 3.1.0 | ## Modules @@ -149,7 +149,6 @@ kubectl get nodes | [amazon\_eks\_aws\_ebs\_csi\_driver\_config](#input\_amazon\_eks\_aws\_ebs\_csi\_driver\_config) | configMap for AWS EBS CSI Driver add-on | `any` | `{}` | no | | [amazon\_eks\_coredns\_config](#input\_amazon\_eks\_coredns\_config) | Configuration for Amazon CoreDNS EKS add-on | `any` | `{}` | no | | [amazon\_eks\_kube\_proxy\_config](#input\_amazon\_eks\_kube\_proxy\_config) | ConfigMap for Amazon EKS Kube-Proxy add-on | `any` | `{}` | no | -| [amazon\_eks\_vpc\_cni](#input\_amazon\_eks\_vpc\_cni) | The VPC CNI add-on configuration.
enable - (Optional) Whether to enable the add-on. Defaults to false.
before\_compute - (Optional) Whether to create the add-on before the compute resources. Defaults to true.
most\_recent - (Optional) Whether to use the most recent version of the add-on. Defaults to true.
resolve\_conflicts - (Optional) How to resolve parameter value conflicts between the add-on and the cluster. Defaults to OVERWRITE. Valid values: OVERWRITE, NONE, PRESERVE.
configuration\_values - (Optional) A map of configuration values for the add-on. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_add-on for supported values.
preserve - (Optional) Whether to preserve the add-on's objects when the add-on is deleted. Defaults to false. | 
object({
    enable               = bool
    before_compute       = bool
    most_recent          = bool
    resolve_conflicts    = string
    configuration_values = map(any) # hcl format later to be json encoded
    preserve             = bool
  })
| 
{
  "before_compute": true,
  "configuration_values": {
    "AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG": "true",
    "ENABLE_PREFIX_DELEGATION": "true",
    "ENI_CONFIG_LABEL_DEF": "topology.kubernetes.io/zone",
    "WARM_PREFIX_TARGET": "1"
  },
  "enable": false,
  "most_recent": true,
  "preserve": false,
  "resolve_conflicts": "OVERWRITE"
}
| no | | [assign\_public\_ip](#input\_assign\_public\_ip) | Whether to assign a public IP to the bastion | `bool` | `false` | no | | [aws\_admin\_usernames](#input\_aws\_admin\_usernames) | A list of one or more AWS usernames with authorized access to KMS and EKS resources, will automatically add the user running the terraform as an admin | `list(string)` | `[]` | no | | [aws\_node\_termination\_handler\_helm\_config](#input\_aws\_node\_termination\_handler\_helm\_config) | AWS Node Termination Handler Helm Chart config | `any` | `{}` | no | @@ -157,17 +156,20 @@ kubectl get nodes | [bastion\_ssh\_password](#input\_bastion\_ssh\_password) | The SSH password to use for the bastion if SSM authentication is used | `string` | `"my-password"` | no | | [bastion\_ssh\_user](#input\_bastion\_ssh\_user) | The SSH user to use for the bastion | `string` | `"ec2-user"` | no | | [bastion\_tenancy](#input\_bastion\_tenancy) | The tenancy of the bastion | `string` | `"default"` | no | +| [calico\_helm\_config](#input\_calico\_helm\_config) | Calico Helm Chart config | `any` | `{}` | no | +| [cluster\_addons](#input\_cluster\_addons) | Nested of eks native add-ons and their associated parameters.
See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_add-on for supported values.
See https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/examples/complete/main.tf#L44-L60 for upstream example.

to see available eks marketplace addons available for your cluster's version run:
aws eks describe-addon-versions --kubernetes-version $k8s\_cluster\_version --query 'addons[].{MarketplaceProductUrl: marketplaceInformation.productUrl, Name: addonName, Owner: owner Publisher: publisher, Type: type}' --output table | `any` | `{}` | no | | [cluster\_autoscaler\_helm\_config](#input\_cluster\_autoscaler\_helm\_config) | Cluster Autoscaler Helm Chart config | `any` | `{}` | no | | [cluster\_endpoint\_public\_access](#input\_cluster\_endpoint\_public\_access) | Whether to enable private access to the EKS cluster | `bool` | `false` | no | | [cluster\_version](#input\_cluster\_version) | Kubernetes version to use for EKS cluster | `string` | `"1.23"` | no | +| [create\_aws\_auth\_configmap](#input\_create\_aws\_auth\_configmap) | Determines whether to create the aws-auth configmap. NOTE - this is only intended for scenarios where the configmap does not exist (i.e. - when using only self-managed node groups). Most users should use `manage_aws_auth_configmap` | `bool` | `false` | no | | [create\_database\_subnet\_group](#input\_create\_database\_subnet\_group) | Whether to create a database subnet group | `bool` | `true` | no | | [create\_database\_subnet\_route\_table](#input\_create\_database\_subnet\_route\_table) | Whether to create a database subnet route table | `bool` | `true` | no | -| [default\_tags](#input\_default\_tags) | A map of default tags to apply to all resources | `map(string)` | `{}` | no | | [eks\_worker\_tenancy](#input\_eks\_worker\_tenancy) | The tenancy of the EKS worker nodes | `string` | `"default"` | no | | [enable\_amazon\_eks\_aws\_ebs\_csi\_driver](#input\_enable\_amazon\_eks\_aws\_ebs\_csi\_driver) | Enable EKS Managed AWS EBS CSI Driver add-on; enable\_amazon\_eks\_aws\_ebs\_csi\_driver and enable\_self\_managed\_aws\_ebs\_csi\_driver are mutually exclusive | `bool` | `false` | no | | [enable\_amazon\_eks\_coredns](#input\_enable\_amazon\_eks\_coredns) | Enable Amazon EKS CoreDNS add-on | `bool` | `false` | no | | [enable\_amazon\_eks\_kube\_proxy](#input\_enable\_amazon\_eks\_kube\_proxy) | Enable Kube Proxy add-on | `bool` | `false` | no | | [enable\_aws\_node\_termination\_handler](#input\_enable\_aws\_node\_termination\_handler) | Enable AWS Node Termination Handler add-on | `bool` | `false` | no | +| [enable\_calico](#input\_enable\_calico) | Enable Calico add-on | `bool` | `true` | no | | [enable\_cluster\_autoscaler](#input\_enable\_cluster\_autoscaler) | Enable Cluster autoscaler add-on | `bool` | `false` | no | | [enable\_eks\_managed\_nodegroups](#input\_enable\_eks\_managed\_nodegroups) | Enable managed node groups | `bool` | n/a | yes | | [enable\_metrics\_server](#input\_enable\_metrics\_server) | Enable metrics server add-on | `bool` | `false` | no | @@ -187,6 +189,7 @@ kubectl get nodes | [name\_prefix](#input\_name\_prefix) | The prefix to use when naming all resources | `string` | `"ex-complete"` | no | | [region](#input\_region) | The AWS region to deploy into | `string` | n/a | yes | | [region2](#input\_region2) | The AWS region to deploy into | `string` | n/a | yes | +| [tags](#input\_tags) | A map of tags to apply to all resources | `map(string)` | `{}` | no | | [vpc\_cidr](#input\_vpc\_cidr) | The CIDR block for the VPC | `string` | n/a | yes | | [zarf\_version](#input\_zarf\_version) | The version of Zarf to use | `string` | `""` | no | diff --git a/examples/complete/fixtures.common.tfvars b/examples/complete/fixtures.common.tfvars index 684b5e36..8db50acd 100644 --- a/examples/complete/fixtures.common.tfvars +++ b/examples/complete/fixtures.common.tfvars @@ -5,11 +5,10 @@ #region = "us-east-2" # target AWS region #region2 = "us-east-1" # RDS backup target AWS region -# default_tags = { -# Environment = "dev" -# Project = "ci-eks" -# Owner = "ci" -# } +tags = { + Environment = "dev" + Project = "du-iac-cicd" +} name_prefix = "ex-complete" manage_aws_auth_configmap = true @@ -34,7 +33,6 @@ cluster_version = "1.23" ############## Big Bang Dependencies ###################### keycloak_enabled = true -# other_addon_enabled = true #################### Keycloak ########################### @@ -47,20 +45,40 @@ kc_db_allocated_storage = 20 kc_db_max_allocated_storage = 100 kc_db_instance_class = "db.t4g.large" -#################### EKS Addon ######################### -amazon_eks_vpc_cni = { - enable = true - before_compute = true - most_recent = true - resolve_conflicts = "OVERWRITE" - preserve = false - configuration_values = { - # Reference https://aws.github.io/aws-eks-best-practices/reliability/docs/networkmanagement/#cni-custom-networking - AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG = "true" - ENI_CONFIG_LABEL_DEF = "topology.kubernetes.io/zone" # allows vpc-cni to use topology labels to determine which subnet to deploy an ENI in - - # Reference docs https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html - ENABLE_PREFIX_DELEGATION = "true" - WARM_PREFIX_TARGET = "1" +# #################### EKS Addon ######################### +# add other "eks native" marketplace addons and configs to this list +cluster_addons = { + vpc-cni = { + most_recent = true + before_compute = true + configuration_values = <<-JSON + { + "env": { + "AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG": "true", + "ENABLE_PREFIX_DELEGATION": "true", + "ENI_CONFIG_LABEL_DEF": "topology.kubernetes.io/zone", + "WARM_PREFIX_TARGET": "1", + "ANNOTATE_POD_IP": "true" + } + } + JSON } } + + +#################### Blueprints addons ################### +#wait false for all addons, as it times out on teardown in the test pipeline +enable_cluster_autoscaler = true +cluster_autoscaler_helm_config = { wait = false } + +enable_amazon_eks_aws_ebs_csi_driver = true +amazon_eks_aws_ebs_csi_driver_config = { wait = false } + +enable_metrics_server = true +metrics_server_helm_config = { wait = false } + +enable_aws_node_termination_handler = true +aws_node_termination_handler_helm_config = { wait = false } + +enable_calico = true +calico_helm_config = { wait = false } diff --git a/examples/complete/fixtures.secure.tfvars b/examples/complete/fixtures.secure.tfvars index 668bc9f5..8f4e77fc 100644 --- a/examples/complete/fixtures.secure.tfvars +++ b/examples/complete/fixtures.secure.tfvars @@ -5,3 +5,5 @@ enable_self_managed_nodegroups = true bastion_tenancy = "dedicated" eks_worker_tenancy = "dedicated" cluster_endpoint_public_access = false + +create_aws_auth_configmap = true #secure example assumes enable_eks_managed_nodegroups = false, need to creaste the configmap ourselves diff --git a/examples/complete/main.tf b/examples/complete/main.tf index 7a3ed48c..6a04c934 100644 --- a/examples/complete/main.tf +++ b/examples/complete/main.tf @@ -17,10 +17,13 @@ locals { account = data.aws_caller_identity.current.account_id - tags = { - Blueprint = replace(basename(path.cwd), "_", "-") # tag names based on the directory name - GithubRepo = "github.com/defenseunicorns/iac" - } + tags = merge( + var.tags, + { + RootTFModule = replace(basename(path.cwd), "_", "-") # tag names based on the directory name + GithubRepo = "github.com/defenseunicorns/iac" + } + ) eks_managed_node_groups = { # Managed Node groups with minimum config @@ -96,10 +99,6 @@ locals { instance_type = "m5.xlarge" #capacity_type = "" # Optional Use this only for SPOT capacity as capacity_type = "spot". Only for eks_managed_node_groups - - tags = { - subnet_type = "private" - } } } } @@ -172,9 +171,9 @@ module "bastion" { vpc_endpoints_enabled = true tenancy = var.bastion_tenancy zarf_version = var.zarf_version - tags = merge(local.tags, { - Function = "bastion-ssm" - }) + tags = merge( + local.tags, + { Function = "bastion-ssm" }) } ########################################################### @@ -198,6 +197,7 @@ module "eks" { bastion_role_name = module.bastion.bastion_role_name # If using EKS Managed Node Groups, the aws-auth ConfigMap is created by eks itself and terraform can not create it + create_aws_auth_configmap = var.create_aws_auth_configmap manage_aws_auth_configmap = var.manage_aws_auth_configmap ######################## EKS Managed Node Group ################################### @@ -223,26 +223,32 @@ module "eks" { AmazonSSMManagedInstanceCore = "arn:${data.aws_partition.current.partition}:iam::aws:policy/AmazonSSMManagedInstanceCore" } # enable discovery of autoscaling groups by cluster-autoscaler - autoscaling_group_tags = { - "k8s.io/cluster-autoscaler/enabled" : true, - "k8s.io/cluster-autoscaler/${local.cluster_name}" : "owned" - } + autoscaling_group_tags = merge( + local.tags, + { + "k8s.io/cluster-autoscaler/enabled" : true, + "k8s.io/cluster-autoscaler/${local.cluster_name}" : "owned" + }) metadata_options = { #https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#metadata-options http_endpoint = "enabled" http_put_response_hop_limit = 2 http_tokens = "optional" # set to "enabled" to enforce IMDSv2, default for upstream terraform-aws-eks module } + tags = { + subnet_type = "private" + } } + tags = local.tags + self_managed_node_groups = local.self_managed_node_groups #--------------------------------------------------------------- #"native" EKS Add-Ons #--------------------------------------------------------------- - # VPC CNI - amazon_eks_vpc_cni = var.amazon_eks_vpc_cni + cluster_addons = var.cluster_addons #--------------------------------------------------------------- # EKS Blueprints - EKS Add-Ons @@ -271,4 +277,8 @@ module "eks" { # EKS Cluster Autoscaler enable_cluster_autoscaler = var.enable_cluster_autoscaler cluster_autoscaler_helm_config = var.cluster_autoscaler_helm_config + + #Calico + enable_calico = var.enable_calico + calico_helm_config = var.calico_helm_config } diff --git a/examples/complete/providers.tf b/examples/complete/providers.tf index 427c1c5c..7f657d68 100644 --- a/examples/complete/providers.tf +++ b/examples/complete/providers.tf @@ -7,7 +7,7 @@ terraform { } aws = { source = "hashicorp/aws" - version = ">= 4.47.0" + version = ">= 4.62.0" } cloudinit = { source = "hashicorp/cloudinit" @@ -51,7 +51,7 @@ terraform { provider "aws" { region = var.region # default_tags { - # tags = var.default_tags + # tags = var.tags #bug https://github.com/hashicorp/terraform-provider-aws/issues/19583#issuecomment-855773246 # } } @@ -59,7 +59,7 @@ provider "aws" { alias = "region2" region = var.region2 # default_tags { - # tags = var.default_tags + # tags = var.tags #bug https://github.com/hashicorp/terraform-provider-aws/issues/19583#issuecomment-855773246 # } } diff --git a/examples/complete/variables.tf b/examples/complete/variables.tf index 78c46a80..c8b78697 100644 --- a/examples/complete/variables.tf +++ b/examples/complete/variables.tf @@ -27,14 +27,20 @@ variable "aws_admin_usernames" { default = [] } +variable "create_aws_auth_configmap" { + description = "Determines whether to create the aws-auth configmap. NOTE - this is only intended for scenarios where the configmap does not exist (i.e. - when using only self-managed node groups). Most users should use `manage_aws_auth_configmap`" + type = bool + default = false +} + variable "manage_aws_auth_configmap" { description = "Determines whether to manage the aws-auth configmap" type = bool default = false } -variable "default_tags" { - description = "A map of default tags to apply to all resources" +variable "tags" { + description = "A map of tags to apply to all resources" type = map(string) default = {} } @@ -113,41 +119,17 @@ variable "enable_self_managed_nodegroups" { ########################################################### ################## EKS Addons Config ###################### -#----------------AWS EKS VPC CNI------------------------- -variable "amazon_eks_vpc_cni" { +variable "cluster_addons" { description = <<-EOD - The VPC CNI add-on configuration. - enable - (Optional) Whether to enable the add-on. Defaults to false. - before_compute - (Optional) Whether to create the add-on before the compute resources. Defaults to true. - most_recent - (Optional) Whether to use the most recent version of the add-on. Defaults to true. - resolve_conflicts - (Optional) How to resolve parameter value conflicts between the add-on and the cluster. Defaults to OVERWRITE. Valid values: OVERWRITE, NONE, PRESERVE. - configuration_values - (Optional) A map of configuration values for the add-on. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_add-on for supported values. - preserve - (Optional) Whether to preserve the add-on's objects when the add-on is deleted. Defaults to false. - EOD - type = object({ - enable = bool - before_compute = bool - most_recent = bool - resolve_conflicts = string - configuration_values = map(any) # hcl format later to be json encoded - preserve = bool - }) - default = { - before_compute = true - enable = false - most_recent = true - resolve_conflicts = "OVERWRITE" - preserve = false - configuration_values = { - # Reference https://aws.github.io/aws-eks-best-practices/reliability/docs/networkmanagement/#cni-custom-networking - AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG = "true" - ENI_CONFIG_LABEL_DEF = "topology.kubernetes.io/zone" # allows vpc-cni to use topology labels to determine which subnet to deploy an ENI in - - # Reference docs https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html - ENABLE_PREFIX_DELEGATION = "true" - WARM_PREFIX_TARGET = "1" - } - } + Nested of eks native add-ons and their associated parameters. + See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_add-on for supported values. + See https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/examples/complete/main.tf#L44-L60 for upstream example. + + to see available eks marketplace addons available for your cluster's version run: + aws eks describe-addon-versions --kubernetes-version $k8s_cluster_version --query 'addons[].{MarketplaceProductUrl: marketplaceInformation.productUrl, Name: addonName, Owner: owner Publisher: publisher, Type: type}' --output table +EOD + type = any + default = {} } #----------------AWS CoreDNS------------------------- @@ -228,6 +210,20 @@ variable "cluster_autoscaler_helm_config" { default = {} } +#----------------Calico------------------------- +variable "enable_calico" { + description = "Enable Calico add-on" + type = bool + default = true +} + +variable "calico_helm_config" { + description = "Calico Helm Chart config" + type = any + default = {} +} + + ########################################################### ################## Bastion Config ######################### variable "bastion_tenancy" { diff --git a/modules/bastion/main.tf b/modules/bastion/main.tf index 08f9c2f6..b07ad32f 100644 --- a/modules/bastion/main.tf +++ b/modules/bastion/main.tf @@ -61,9 +61,10 @@ resource "aws_instance" "application" { subnet_id = var.subnet_name != "" ? data.aws_subnet.subnet_by_name[0].id : var.subnet_id - tags = { - Name = var.name - } + tags = merge( + var.tags, + { Name = var.name } + ) } resource "tls_private_key" "bastion_key" { diff --git a/modules/eks/README.md b/modules/eks/README.md index 79c4f247..94e6414b 100644 --- a/modules/eks/README.md +++ b/modules/eks/README.md @@ -51,7 +51,6 @@ To view examples for how you can leverage this EKS Module, please see the [examp | [amazon\_eks\_aws\_ebs\_csi\_driver\_config](#input\_amazon\_eks\_aws\_ebs\_csi\_driver\_config) | configMap for AWS EBS CSI Driver add-on | `any` | `{}` | no | | [amazon\_eks\_coredns\_config](#input\_amazon\_eks\_coredns\_config) | Configuration for Amazon CoreDNS EKS add-on | `any` | `{}` | no | | [amazon\_eks\_kube\_proxy\_config](#input\_amazon\_eks\_kube\_proxy\_config) | ConfigMap for Amazon EKS Kube-Proxy add-on | `any` | `{}` | no | -| [amazon\_eks\_vpc\_cni](#input\_amazon\_eks\_vpc\_cni) | The VPC CNI add-on configuration.
enable - (Optional) Whether to enable the add-on. Defaults to false.
before\_compute - (Optional) Whether to create the add-on before the compute resources. Defaults to true.
most\_recent - (Optional) Whether to use the most recent version of the add-on. Defaults to true.
resolve\_conflicts - (Optional) How to resolve parameter value conflicts between the add-on and the cluster. Defaults to OVERWRITE. Valid values: OVERWRITE, NONE, PRESERVE.
configuration\_values - (Optional) A map of configuration values for the add-on. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_add-on for supported values.
preserve - (Optional) Whether to preserve the add-on's objects when the add-on is deleted. Defaults to false. | 
object({
    enable               = bool
    before_compute       = bool
    most_recent          = bool
    resolve_conflicts    = string
    configuration_values = map(any) # hcl format later to be json encoded
    preserve             = bool
  })
| 
{
  "before_compute": true,
  "configuration_values": {
    "AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG": "true",
    "ENABLE_PREFIX_DELEGATION": "true",
    "ENI_CONFIG_LABEL_DEF": "topology.kubernetes.io/zone",
    "WARM_PREFIX_TARGET": "1"
  },
  "enable": false,
  "most_recent": true,
  "preserve": false,
  "resolve_conflicts": "OVERWRITE"
}
| no | | [aws\_account](#input\_aws\_account) | n/a | `string` | `""` | no | | [aws\_admin\_usernames](#input\_aws\_admin\_usernames) | A list of one or more AWS usernames with authorized access to KMS and EKS resources, will automatically add the user running the terraform as an admin | `list(string)` | `[]` | no | | [aws\_auth\_users](#input\_aws\_auth\_users) | List of map of users to add to aws-auth configmap | 
list(object({
    userarn  = string
    username = string
    groups   = list(string)
  }))
| `[]` | no | @@ -59,18 +58,22 @@ To view examples for how you can leverage this EKS Module, please see the [examp | [aws\_region](#input\_aws\_region) | n/a | `string` | `""` | no | | [bastion\_role\_arn](#input\_bastion\_role\_arn) | ARN of role authorized kubectl access | `string` | `""` | no | | [bastion\_role\_name](#input\_bastion\_role\_name) | Name of role authorized kubectl access | `string` | `""` | no | +| [calico\_helm\_config](#input\_calico\_helm\_config) | Calico Helm Chart config | `any` | `{}` | no | +| [cluster\_addons](#input\_cluster\_addons) | Nested of eks native add-ons and their associated parameters.
See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_add-on for supported values.
See https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/examples/complete/main.tf#L44-L60 for upstream example.

to see available eks marketplace addons available for your cluster's version run:
aws eks describe-addon-versions --kubernetes-version $k8s\_cluster\_version --query 'addons[].{MarketplaceProductUrl: marketplaceInformation.productUrl, Name: addonName, Owner: owner Publisher: publisher, Type: type}' --output table | `any` | `{}` | no | | [cluster\_autoscaler\_helm\_config](#input\_cluster\_autoscaler\_helm\_config) | Cluster Autoscaler Helm Chart config | `any` | 
{
  "set": [
    {
      "name": "extraArgs.expander",
      "value": "priority"
    },
    {
      "name": "expanderPriorities",
      "value": "100:\n  - .*-spot-2vcpu-8mem.*\n90:\n  - .*-spot-4vcpu-16mem.*\n10:\n  - .*\n"
    }
  ]
}
| no | | [cluster\_endpoint\_private\_access](#input\_cluster\_endpoint\_private\_access) | Enable private access to the cluster endpoint | `bool` | `true` | no | | [cluster\_endpoint\_public\_access](#input\_cluster\_endpoint\_public\_access) | Enable public access to the cluster endpoint | `bool` | `false` | no | | [cluster\_name](#input\_cluster\_name) | Name of cluster - used by Terratest for e2e test automation | `string` | `""` | no | | [cluster\_version](#input\_cluster\_version) | Kubernetes version to use for EKS cluster | `string` | `"1.23"` | no | | [control\_plane\_subnet\_ids](#input\_control\_plane\_subnet\_ids) | Subnet IDs for control plane | `list(string)` | `[]` | no | +| [create\_aws\_auth\_configmap](#input\_create\_aws\_auth\_configmap) | Determines whether to create the aws-auth configmap. NOTE - this is only intended for scenarios where the configmap does not exist (i.e. - when using only self-managed node groups). Most users should use `manage_aws_auth_configmap` | `bool` | `false` | no | | [eks\_managed\_node\_group\_defaults](#input\_eks\_managed\_node\_group\_defaults) | Map of EKS-managed node group default configurations | `any` | `{}` | no | | [eks\_managed\_node\_groups](#input\_eks\_managed\_node\_groups) | Managed node groups configuration | `any` | `{}` | no | | [enable\_amazon\_eks\_aws\_ebs\_csi\_driver](#input\_enable\_amazon\_eks\_aws\_ebs\_csi\_driver) | Enable EKS Managed AWS EBS CSI Driver add-on; enable\_amazon\_eks\_aws\_ebs\_csi\_driver and enable\_self\_managed\_aws\_ebs\_csi\_driver are mutually exclusive | `bool` | `false` | no | | [enable\_amazon\_eks\_coredns](#input\_enable\_amazon\_eks\_coredns) | Enable Amazon EKS CoreDNS add-on | `bool` | `false` | no | | [enable\_amazon\_eks\_kube\_proxy](#input\_enable\_amazon\_eks\_kube\_proxy) | Enable Kube Proxy add-on | `bool` | `false` | no | | [enable\_aws\_node\_termination\_handler](#input\_enable\_aws\_node\_termination\_handler) | Enable AWS Node Termination Handler add-on | `bool` | `false` | no | +| [enable\_calico](#input\_enable\_calico) | Enable Calico add-on | `bool` | `false` | no | | [enable\_cluster\_autoscaler](#input\_enable\_cluster\_autoscaler) | Enable Cluster autoscaler add-on | `bool` | `false` | no | | [enable\_metrics\_server](#input\_enable\_metrics\_server) | Enable metrics server add-on | `bool` | `false` | no | | [kms\_key\_administrators](#input\_kms\_key\_administrators) | List of ARNs of additional administrator users to add to KMS key policy | `list(string)` | `[]` | no | @@ -82,6 +85,7 @@ To view examples for how you can leverage this EKS Module, please see the [examp | [self\_managed\_node\_group\_defaults](#input\_self\_managed\_node\_group\_defaults) | Map of self-managed node group default configurations | `any` | `{}` | no | | [self\_managed\_node\_groups](#input\_self\_managed\_node\_groups) | Self-managed node groups configuration | `any` | `{}` | no | | [source\_security\_group\_id](#input\_source\_security\_group\_id) | List of additional rules to add to cluster security group | `string` | `""` | no | +| [tags](#input\_tags) | A map of tags to apply to all resources | `map(string)` | `{}` | no | | [tenancy](#input\_tenancy) | Tenancy of the cluster | `string` | `"dedicated"` | no | | [vpc\_cni\_custom\_subnet](#input\_vpc\_cni\_custom\_subnet) | Subnet to put pod ENIs in | `list(string)` | `[]` | no | | [vpc\_id](#input\_vpc\_id) | VPC ID | `string` | `""` | no | diff --git a/modules/eks/eks-addons.tf b/modules/eks/eks-addons.tf index eebb56b2..6fb73676 100644 --- a/modules/eks/eks-addons.tf +++ b/modules/eks/eks-addons.tf @@ -2,14 +2,21 @@ # EKS Add-Ons #--------------------------------------------------------------- +locals { + self_managed_node_group_names = [for key, value in module.aws_eks.self_managed_node_groups : lookup(value, "autoscaling_group_name", "")] +} + module "eks_blueprints_kubernetes_addons" { source = "git::https://github.com/aws-ia/terraform-aws-eks-blueprints.git//modules/kubernetes-addons?ref=v4.27.0" - eks_cluster_id = module.aws_eks.cluster_name - eks_cluster_endpoint = module.aws_eks.cluster_endpoint - eks_oidc_provider = module.aws_eks.oidc_provider - eks_cluster_version = module.aws_eks.cluster_version - auto_scaling_group_names = concat(lookup(module.aws_eks.self_managed_node_groups, "autoscaling_group_name", []), lookup(module.aws_eks.eks_managed_node_groups, "node_group_autoscaling_group_names", [])) + eks_cluster_id = module.aws_eks.cluster_name + eks_cluster_endpoint = module.aws_eks.cluster_endpoint + eks_oidc_provider = module.aws_eks.oidc_provider + eks_cluster_version = module.aws_eks.cluster_version + + # only used for aws_node_termination_handler, if this list is empty, then enable_aws_node_termination_handler should also be false. + auto_scaling_group_names = local.self_managed_node_group_names + # blueprints addons # EKS CoreDNS enable_amazon_eks_coredns = var.enable_amazon_eks_coredns @@ -23,8 +30,6 @@ module "eks_blueprints_kubernetes_addons" { enable_amazon_eks_aws_ebs_csi_driver = var.enable_amazon_eks_aws_ebs_csi_driver amazon_eks_aws_ebs_csi_driver_config = var.amazon_eks_aws_ebs_csi_driver_config - - # K8s Add-ons # EKS Metrics Server enable_metrics_server = var.enable_metrics_server metrics_server_helm_config = var.metrics_server_helm_config @@ -36,4 +41,10 @@ module "eks_blueprints_kubernetes_addons" { # EKS Cluster Autoscaler enable_cluster_autoscaler = var.enable_cluster_autoscaler cluster_autoscaler_helm_config = var.cluster_autoscaler_helm_config + + # Calico + enable_calico = var.enable_calico + calico_helm_config = var.calico_helm_config + + tags = var.tags } diff --git a/modules/eks/k8s-manifests.tf b/modules/eks/k8s-manifests.tf index 75b7ae21..d42bd9ab 100644 --- a/modules/eks/k8s-manifests.tf +++ b/modules/eks/k8s-manifests.tf @@ -11,10 +11,9 @@ kind: ENIConfig metadata: name: ${each.key} spec: - subnet : ${each.value} - securityGroups : - - ${module.aws_eks.cluster_primary_security_group_id} - - ${module.aws_eks.node_security_group_id} + subnet: ${each.value} + securityGroups: + ${indent(4, join("\n", [for sg in compact([module.aws_eks.cluster_primary_security_group_id, module.aws_eks.node_security_group_id, module.aws_eks.cluster_security_group_id]) : "- ${sg}"]))} YAML depends_on = [ diff --git a/modules/eks/locals.tf b/modules/eks/locals.tf index 336b5258..d05a388d 100644 --- a/modules/eks/locals.tf +++ b/modules/eks/locals.tf @@ -8,39 +8,9 @@ locals { [for admin_user in var.aws_admin_usernames : "arn:${data.aws_partition.current.partition}:iam::${data.aws_caller_identity.current.account_id}:user/${admin_user}"], [data.aws_caller_identity.current.arn] )) - aws_auth_users = distinct(concat([for admin_user in var.aws_admin_usernames : { + aws_auth_users = [for admin_user in var.aws_admin_usernames : { userarn = "arn:${data.aws_partition.current.partition}:iam::${data.aws_caller_identity.current.account_id}:user/${admin_user}" username = admin_user groups = ["system:masters"] - }], - [{ - userarn = data.aws_caller_identity.current.arn - username = split("/", data.aws_caller_identity.current.arn)[1] - groups = ["system:masters"] - }] - )) - - # if using EKS Managed Node Groups you can not also create the aws-auth configmap because eks does it for you - it will already exist when TF tries to create it and you will receive an error. - # the following logic determines if the aws-auth configmap should be created or not by checking if eks_managed_node_groups would be created based on inputs to the upstream eks module - # this returns true (will create the configmap) if eks_managed_node_groups is empty or if eks_managed_node_groups is not empty AND all eks_managed_node_groups.*.create values are set to false - # it returns false (won't create the configmap) when eks_managed_node_groups is not empty AND at least one eks_managed_node_groups.*.create value is set to true OR is not defined - create_aws_auth_configmap = !( - # Check if eks_managed_node_groups is not empty - length(var.eks_managed_node_groups) > 0 && ( - # Check if any EKS managed node group value is set to create or not defined, if not defined, then set to true as null = true in upstream. - length([for v in values(var.eks_managed_node_groups) : v if try(v.create, true) == true]) > 0 || - # Check if all EKS managed node groups have create set to false - length([for v in values(var.eks_managed_node_groups) : v if try(v.create, true) == false]) < length(var.eks_managed_node_groups) - ) - ) - - cluster_addons = { - vpc-cni = lookup(var.amazon_eks_vpc_cni, "enable", null) ? { - before_compute = lookup(var.amazon_eks_vpc_cni, "before_compute", null) - most_recent = lookup(var.amazon_eks_vpc_cni, "most_recent", null) - configuration_values = jsonencode({ env = (lookup(var.amazon_eks_vpc_cni, "configuration_values", null)) }) - resolve_conflicts = lookup(var.amazon_eks_vpc_cni, "resolve_conflicts", null) - preserve = lookup(var.amazon_eks_vpc_cni, "preserve", null) - } : null - } + }] } diff --git a/modules/eks/main.tf b/modules/eks/main.tf index 04e0843d..ddac2883 100644 --- a/modules/eks/main.tf +++ b/modules/eks/main.tf @@ -18,7 +18,7 @@ module "aws_eks" { self_managed_node_groups = var.self_managed_node_groups eks_managed_node_groups = var.eks_managed_node_groups - cluster_addons = local.cluster_addons + cluster_addons = var.cluster_addons #----------------------------------------------------------------------------------------------------------# # Security groups used in this module created by the upstream modules terraform-aws-eks (https://github.com/terraform-aws-modules/terraform-aws-eks). @@ -39,7 +39,7 @@ module "aws_eks" { } } - create_aws_auth_configmap = local.create_aws_auth_configmap + create_aws_auth_configmap = var.create_aws_auth_configmap manage_aws_auth_configmap = var.manage_aws_auth_configmap kms_key_administrators = distinct(concat(local.admin_arns, var.kms_key_administrators)) @@ -56,6 +56,8 @@ module "aws_eks" { groups = ["system:masters"] } ] + + tags = var.tags } resource "aws_iam_role" "auth_eks_role" { diff --git a/modules/eks/variables.tf b/modules/eks/variables.tf index 86c107be..fbac1086 100644 --- a/modules/eks/variables.tf +++ b/modules/eks/variables.tf @@ -15,6 +15,12 @@ variable "cluster_version" { } } +variable "tags" { + description = "A map of tags to apply to all resources" + type = map(string) + default = {} +} + variable "vpc_id" { description = "VPC ID" type = string @@ -70,6 +76,12 @@ variable "aws_admin_usernames" { default = [] } +variable "create_aws_auth_configmap" { + description = "Determines whether to create the aws-auth configmap. NOTE - this is only intended for scenarios where the configmap does not exist (i.e. - when using only self-managed node groups). Most users should use `manage_aws_auth_configmap`" + type = bool + default = false +} + variable "manage_aws_auth_configmap" { description = "Determines whether to manage the aws-auth configmap" type = bool @@ -153,42 +165,19 @@ variable "eks_managed_node_group_defaults" { } ########################################################### -################## EKS add-ons Config ###################### +################## EKS "Native" add-ons Config ###################### -variable "amazon_eks_vpc_cni" { +variable "cluster_addons" { description = <<-EOD - The VPC CNI add-on configuration. - enable - (Optional) Whether to enable the add-on. Defaults to false. - before_compute - (Optional) Whether to create the add-on before the compute resources. Defaults to true. - most_recent - (Optional) Whether to use the most recent version of the add-on. Defaults to true. - resolve_conflicts - (Optional) How to resolve parameter value conflicts between the add-on and the cluster. Defaults to OVERWRITE. Valid values: OVERWRITE, NONE, PRESERVE. - configuration_values - (Optional) A map of configuration values for the add-on. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_add-on for supported values. - preserve - (Optional) Whether to preserve the add-on's objects when the add-on is deleted. Defaults to false. - EOD - type = object({ - enable = bool - before_compute = bool - most_recent = bool - resolve_conflicts = string - configuration_values = map(any) # hcl format later to be json encoded - preserve = bool - }) - default = { - before_compute = true - enable = false - most_recent = true - resolve_conflicts = "OVERWRITE" - preserve = false - configuration_values = { - # Reference https://aws.github.io/aws-eks-best-practices/reliability/docs/networkmanagement/#cni-custom-networking - AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG = "true" - ENI_CONFIG_LABEL_DEF = "topology.kubernetes.io/zone" # allows vpc-cni to use topology labels to determine which subnet to deploy an ENI in - - # Reference docs https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html - ENABLE_PREFIX_DELEGATION = "true" - WARM_PREFIX_TARGET = "1" - } - } + Nested of eks native add-ons and their associated parameters. + See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_add-on for supported values. + See https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/examples/complete/main.tf#L44-L60 for upstream example. + + to see available eks marketplace addons available for your cluster's version run: + aws eks describe-addon-versions --kubernetes-version $k8s_cluster_version --query 'addons[].{MarketplaceProductUrl: marketplaceInformation.productUrl, Name: addonName, Owner: owner Publisher: publisher, Type: type}' --output table +EOD + type = any + default = {} } #----------------AWS CoreDNS------------------------- @@ -286,3 +275,17 @@ variable "cluster_autoscaler_helm_config" { ] } } + +#----------------Calico------------------------- + +variable "enable_calico" { + description = "Enable Calico add-on" + type = bool + default = false +} + +variable "calico_helm_config" { + description = "Calico Helm Chart config" + type = any + default = {} +} diff --git a/modules/rds/README.md b/modules/rds/README.md index afa45ac6..78f1da94 100644 --- a/modules/rds/README.md +++ b/modules/rds/README.md @@ -53,7 +53,7 @@ To view examples for how you can leverage this RDS Module, please see the [examp | [major\_engine\_version](#input\_major\_engine\_version) | The major version of the engine that this option group should be associated with. | `string` | `""` | no | | [max\_allocated\_storage](#input\_max\_allocated\_storage) | The upper limit to which Amazon RDS can automatically scale the storage of the DB instance. | `number` | `0` | no | | [password](#input\_password) | Password for the master DB user. Note that this may show up in logs, and it will be stored in the state file.
The password provided will not be used if the variable create\_random\_password is set to true. | `string` | `null` | no | -| [tags](#input\_tags) | A mapping of tags to assign to the resource. | `map(string)` | `{}` | no | +| [tags](#input\_tags) | A map of tags to apply to all resources | `map(string)` | `{}` | no | | [username](#input\_username) | Username for the master DB user. | `string` | `""` | no | | [vpc\_cidr](#input\_vpc\_cidr) | The CIDR block for the VPC. | `string` | `""` | no | | [vpc\_id](#input\_vpc\_id) | The ID of the VPC. | `string` | `""` | no | diff --git a/modules/rds/variables.tf b/modules/rds/variables.tf index 7117868a..d89158c8 100644 --- a/modules/rds/variables.tf +++ b/modules/rds/variables.tf @@ -77,7 +77,7 @@ variable "max_allocated_storage" { } variable "tags" { - description = "A mapping of tags to assign to the resource." + description = "A map of tags to apply to all resources" type = map(string) default = {} } diff --git a/modules/s3-irsa/README.md b/modules/s3-irsa/README.md index 43edb03f..f5e5abba 100644 --- a/modules/s3-irsa/README.md +++ b/modules/s3-irsa/README.md @@ -64,7 +64,7 @@ To view examples for how you can leverage this S3-IRSA Module, please see the [e | [name\_prefix](#input\_name\_prefix) | Name prefix for all resources that use a randomized suffix | `string` | n/a | yes | | [policy\_name\_prefix](#input\_policy\_name\_prefix) | IAM Policy name prefix | `string` | `"irsa-policy"` | no | | [region](#input\_region) | AWS Region | `string` | `""` | no | -| [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit`,`XYZ`) | `map(string)` | `{}` | no | +| [tags](#input\_tags) | A map of tags to apply to all resources | `map(string)` | `{}` | no | ## Outputs diff --git a/modules/s3-irsa/variables.tf b/modules/s3-irsa/variables.tf index f25b4e50..4ab2c583 100644 --- a/modules/s3-irsa/variables.tf +++ b/modules/s3-irsa/variables.tf @@ -44,7 +44,7 @@ variable "eks_oidc_provider_arn" { } variable "tags" { - description = "Additional tags (e.g. `map('BusinessUnit`,`XYZ`)" + description = "A map of tags to apply to all resources" type = map(string) default = {} } diff --git a/modules/sops/README.md b/modules/sops/README.md index a8636e14..1cc1dc16 100644 --- a/modules/sops/README.md +++ b/modules/sops/README.md @@ -50,7 +50,7 @@ No modules. | [region](#input\_region) | AWS region | `string` | n/a | yes | | [role\_name](#input\_role\_name) | Role to attach the sops policy to | `string` | `""` | no | | [sops\_iam\_policies](#input\_sops\_iam\_policies) | IAM Policies for IRSA IAM role | `list(string)` | `[]` | no | -| [tags](#input\_tags) | Tags to apply to resources | `map(string)` | `{}` | no | +| [tags](#input\_tags) | A map of tags to apply to all resources | `map(string)` | `{}` | no | | [vpc\_id](#input\_vpc\_id) | VPC ID | `string` | `""` | no | ## Outputs diff --git a/modules/sops/variables.tf b/modules/sops/variables.tf index 2838940f..aa155039 100644 --- a/modules/sops/variables.tf +++ b/modules/sops/variables.tf @@ -15,8 +15,8 @@ variable "policy_name_prefix" { } variable "tags" { + description = "A map of tags to apply to all resources" type = map(string) - description = "Tags to apply to resources" default = {} } diff --git a/modules/vpc/README.md b/modules/vpc/README.md index c770c556..d2a272f4 100644 --- a/modules/vpc/README.md +++ b/modules/vpc/README.md @@ -56,6 +56,7 @@ To view examples for how you can leverage this VPC Module, please see the [examp | [public\_subnets](#input\_public\_subnets) | List of public subnets inside the VPC | `list(string)` | `[]` | no | | [region](#input\_region) | AWS region to deploy into | `string` | n/a | yes | | [single\_nat\_gateway](#input\_single\_nat\_gateway) | Use a single NAT gateway for all private subnets | `bool` | `true` | no | +| [tags](#input\_tags) | A map of tags to apply to all resources | `map(string)` | `{}` | no | | [vpc\_cidr](#input\_vpc\_cidr) | CIDR block for the VPC | `string` | n/a | yes | ## Outputs diff --git a/modules/vpc/main.tf b/modules/vpc/main.tf index 501fae91..abd1b5e2 100644 --- a/modules/vpc/main.tf +++ b/modules/vpc/main.tf @@ -41,11 +41,12 @@ data "aws_iam_policy_document" "ecr" { locals { - tags = { - Example = var.name - GithubRepo = "terraform-aws-vpc" - GithubOrg = "terraform-aws-modules" - } + tags = merge( + var.tags, + { + GithubRepo = "terraform-aws-vpc" + GithubOrg = "terraform-aws-modules" + }) } ################################################################################ diff --git a/modules/vpc/variables.tf b/modules/vpc/variables.tf index 7132974b..d75b9b5f 100644 --- a/modules/vpc/variables.tf +++ b/modules/vpc/variables.tf @@ -8,6 +8,12 @@ variable "name" { type = string } +variable "tags" { + description = "A map of tags to apply to all resources" + type = map(string) + default = {} +} + variable "vpc_cidr" { description = "CIDR block for the VPC" type = string