Skip to content

Commit

Permalink
feat: Propagate ami_type to self-managed node group; allow using `a…
Browse files Browse the repository at this point in the history
…mi_type` only (#3030)
  • Loading branch information
bryantbiggs authored May 8, 2024
1 parent afadb14 commit 74d3918
Show file tree
Hide file tree
Showing 30 changed files with 239 additions and 98 deletions.
6 changes: 4 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
rev: v1.88.2
rev: v1.89.1
hooks:
- id: terraform_fmt
- id: terraform_docs
Expand All @@ -24,8 +24,10 @@ repos:
- '--args=--only=terraform_unused_required_providers'
- id: terraform_validate
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v4.6.0
hooks:
- id: check-merge-conflict
- id: end-of-file-fixer
- id: trailing-whitespace
- id: mixed-line-ending
args: [--fix=lf]
9 changes: 3 additions & 6 deletions docs/compute_resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ Refer to the [EKS Managed Node Group documentation](https://docs.aws.amazon.com/
use_custom_launch_template = false
ami_type = "BOTTLEROCKET_x86_64"
platform = "bottlerocket"
}
}
```
Expand All @@ -42,7 +41,6 @@ Refer to the [EKS Managed Node Group documentation](https://docs.aws.amazon.com/
eks_managed_node_groups = {
bottlerocket_prepend_userdata = {
ami_type = "BOTTLEROCKET_x86_64"
platform = "bottlerocket"
bootstrap_extra_args = <<-EOT
# extra args added
Expand Down Expand Up @@ -84,7 +82,7 @@ Refer to the [EKS Managed Node Group documentation](https://docs.aws.amazon.com/
eks_managed_node_groups = {
bottlerocket_custom_ami = {
ami_id = "ami-0ff61e0bcfc81dc94"
platform = "bottlerocket"
ami_type = "BOTTLEROCKET_x86_64"
# use module user data template to bootstrap
enable_bootstrap_user_data = true
Expand Down Expand Up @@ -123,15 +121,15 @@ Refer to the [Self Managed Node Group documentation](https://docs.aws.amazon.com
}
```

2. To use Bottlerocket, specify the `platform` as `bottlerocket` and supply a Bottlerocket OS AMI:
2. To use Bottlerocket, specify the `ami_type` as one of the respective `"BOTTLEROCKET_*" types` and supply a Bottlerocket OS AMI:

```hcl
cluster_version = "1.27"
self_managed_node_groups = {
bottlerocket = {
platform = "bottlerocket"
ami_id = data.aws_ami.bottlerocket_ami.id
ami_type = "BOTTLEROCKET_x86_64"
}
}
```
Expand Down Expand Up @@ -177,7 +175,6 @@ For example, the following creates 4 AWS EKS Managed Node Groups:
# This overrides the OS used
bottlerocket = {
ami_type = "BOTTLEROCKET_x86_64"
platform = "bottlerocket"
}
}
```
18 changes: 9 additions & 9 deletions docs/user_data.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ Users can see the various methods of using and providing user data through the [
- AWS EKS Managed Node Groups
- By default, any supplied user data is pre-pended to the user data supplied by the EKS Managed Node Group service
- If users supply an `ami_id`, the service no longers supplies user data to bootstrap nodes; users can enable `enable_bootstrap_user_data` and use the module provided user data template, or provide their own user data template
- `bottlerocket` platform user data must be in TOML format
- `windows` platform user data must be in powershell/PS1 script format
- AMI types of `BOTTLEROCKET_*`, user data must be in TOML format
- AMI types of `WINDOWS_*`, user data must be in powershell/PS1 script format
- Self Managed Node Groups
- `linux` platform (default) -> the user data template (bash/shell script) provided by the module is used as the default; users are able to provide their own user data template
- `bottlerocket` platform -> the user data template (TOML file) provided by the module is used as the default; users are able to provide their own user data template
- `windows` platform -> the user data template (powershell/PS1 script) provided by the module is used as the default; users are able to provide their own user data template
- `AL2_x86_64` AMI type (default) -> the user data template (bash/shell script) provided by the module is used as the default; users are able to provide their own user data template
- `BOTTLEROCKET_*` AMI types -> the user data template (TOML file) provided by the module is used as the default; users are able to provide their own user data template
- `WINDOWS_*` AMI types -> the user data template (powershell/PS1 script) provided by the module is used as the default; users are able to provide their own user data template

The templates provided by the module can be found under the [templates directory](https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/templates)

Expand All @@ -37,7 +37,7 @@ When using an EKS managed node group, users have 2 primary routes for interactin
bootstrap_extra_args = "..."
post_bootstrap_user_data = "..."
```
- If the AMI is **NOT** an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the `templatefile()` for the respective platform/OS are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.
- If the AMI is **NOT** an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the `templatefile()` for the respective AMI type are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.
- Users can use the following variables to facilitate this process:
```hcl
user_data_template_path = "./your/user_data.sh" # user supplied bootstrap user data template
Expand All @@ -46,12 +46,12 @@ When using an EKS managed node group, users have 2 primary routes for interactin
post_bootstrap_user_data = "..."
```
| ℹ️ When using bottlerocket as the desired platform, since the user data for bottlerocket is TOML, all configurations are merged in the one file supplied as user data. Therefore, `pre_bootstrap_user_data` and `post_bootstrap_user_data` are not valid since the bottlerocket OS handles when various settings are applied. If you wish to supply additional configuration settings when using bottlerocket, supply them via the `bootstrap_extra_args` variable. For the linux platform, `bootstrap_extra_args` are settings that will be supplied to the [AWS EKS Optimized AMI bootstrap script](https://github.com/awslabs/amazon-eks-ami/blob/master/files/bootstrap.sh#L14) such as kubelet extra args, etc. See the [bottlerocket GitHub repository documentation](https://github.com/bottlerocket-os/bottlerocket#description-of-settings) for more details on what settings can be supplied via the `bootstrap_extra_args` variable. |
| ℹ️ When using bottlerocket, the supplied user data (TOML format) is merged in with the values supplied by EKS. Therefore, `pre_bootstrap_user_data` and `post_bootstrap_user_data` are not valid since the bottlerocket OS handles when various settings are applied. If you wish to supply additional configuration settings when using bottlerocket, supply them via the `bootstrap_extra_args` variable. For the `AL2_*` AMI types, `bootstrap_extra_args` are settings that will be supplied to the [AWS EKS Optimized AMI bootstrap script](https://github.com/awslabs/amazon-eks-ami/blob/master/files/bootstrap.sh#L14) such as kubelet extra args, etc. See the [bottlerocket GitHub repository documentation](https://github.com/bottlerocket-os/bottlerocket#description-of-settings) for more details on what settings can be supplied via the `bootstrap_extra_args` variable. |
| :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
### Self Managed Node Group
Self managed node groups require users to provide the necessary bootstrap user data. Users can elect to use the user data template provided by the module for their platform/OS or provide their own user data template for rendering by the module.
Self managed node groups require users to provide the necessary bootstrap user data. Users can elect to use the user data template provided by the module for their respective AMI type or provide their own user data template for rendering by the module.
- If the AMI used is a derivative of the [AWS EKS Optimized AMI ](https://github.com/awslabs/amazon-eks-ami), users can opt in to using a template provided by the module that provides the minimum necessary configuration to bootstrap the node when launched:
- Users can use the following variables to facilitate this process:
Expand All @@ -61,7 +61,7 @@ Self managed node groups require users to provide the necessary bootstrap user d
bootstrap_extra_args = "..."
post_bootstrap_user_data = "..."
```
- If the AMI is **NOT** an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the `templatefile()` for the respective platform/OS are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.
- If the AMI is **NOT** an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the `templatefile()` for the respective AMI type are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.
- Users can use the following variables to facilitate this process:
```hcl
user_data_template_path = "./your/user_data.sh" # user supplied bootstrap user data template
Expand Down
2 changes: 2 additions & 0 deletions examples/eks_managed_node_group/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,14 @@ No inputs.
| <a name="output_cluster_iam_role_unique_id"></a> [cluster\_iam\_role\_unique\_id](#output\_cluster\_iam\_role\_unique\_id) | Stable and unique string identifying the IAM role |
| <a name="output_cluster_id"></a> [cluster\_id](#output\_cluster\_id) | The ID of the EKS cluster. Note: currently a value is returned only for local EKS clusters created on Outposts |
| <a name="output_cluster_identity_providers"></a> [cluster\_identity\_providers](#output\_cluster\_identity\_providers) | Map of attribute maps for all EKS identity providers enabled |
| <a name="output_cluster_ip_family"></a> [cluster\_ip\_family](#output\_cluster\_ip\_family) | The IP family used by the cluster (e.g. `ipv4` or `ipv6`) |
| <a name="output_cluster_name"></a> [cluster\_name](#output\_cluster\_name) | The name of the EKS cluster |
| <a name="output_cluster_oidc_issuer_url"></a> [cluster\_oidc\_issuer\_url](#output\_cluster\_oidc\_issuer\_url) | The URL on the EKS cluster for the OpenID Connect identity provider |
| <a name="output_cluster_platform_version"></a> [cluster\_platform\_version](#output\_cluster\_platform\_version) | Platform version for the cluster |
| <a name="output_cluster_primary_security_group_id"></a> [cluster\_primary\_security\_group\_id](#output\_cluster\_primary\_security\_group\_id) | Cluster security group that was created by Amazon EKS for the cluster. Managed node groups use this security group for control-plane-to-data-plane communication. Referred to as 'Cluster security group' in the EKS console |
| <a name="output_cluster_security_group_arn"></a> [cluster\_security\_group\_arn](#output\_cluster\_security\_group\_arn) | Amazon Resource Name (ARN) of the cluster security group |
| <a name="output_cluster_security_group_id"></a> [cluster\_security\_group\_id](#output\_cluster\_security\_group\_id) | ID of the cluster security group |
| <a name="output_cluster_service_cidr"></a> [cluster\_service\_cidr](#output\_cluster\_service\_cidr) | The CIDR block where Kubernetes pod and service IP addresses are assigned from |
| <a name="output_cluster_status"></a> [cluster\_status](#output\_cluster\_status) | Status of the EKS cluster. One of `CREATING`, `ACTIVE`, `DELETING`, `FAILED` |
| <a name="output_cluster_tls_certificate_sha1_fingerprint"></a> [cluster\_tls\_certificate\_sha1\_fingerprint](#output\_cluster\_tls\_certificate\_sha1\_fingerprint) | The SHA1 fingerprint of the public key of the cluster's certificate |
| <a name="output_eks_managed_node_groups"></a> [eks\_managed\_node\_groups](#output\_eks\_managed\_node\_groups) | Map of attribute maps for all EKS managed node groups created |
Expand Down
8 changes: 2 additions & 6 deletions examples/eks_managed_node_group/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,6 @@ module "eks" {
# to join nodes to the cluster (instead of /etc/eks/bootstrap.sh)
al2023_nodeadm = {
ami_type = "AL2023_x86_64_STANDARD"
platform = "al2023"

use_latest_ami_release_version = true

Expand Down Expand Up @@ -119,13 +118,11 @@ module "eks" {
use_custom_launch_template = false

ami_type = "BOTTLEROCKET_x86_64"
platform = "bottlerocket"
}

# Adds to the AWS provided user data
bottlerocket_add = {
ami_type = "BOTTLEROCKET_x86_64"
platform = "bottlerocket"

use_latest_ami_release_version = true

Expand All @@ -141,7 +138,7 @@ module "eks" {
bottlerocket_custom = {
# Current bottlerocket AMI
ami_id = data.aws_ami.eks_default_bottlerocket.image_id
platform = "bottlerocket"
ami_type = "BOTTLEROCKET_x86_64"

# Use module user data template to bootstrap
enable_bootstrap_user_data = true
Expand Down Expand Up @@ -285,8 +282,8 @@ module "eks" {
# Can be enabled when appropriate for testing/validation
create = false

instance_types = ["trn1n.32xlarge"]
ami_type = "AL2_x86_64_GPU"
instance_types = ["trn1n.32xlarge"]

enable_efa_support = true
pre_bootstrap_user_data = <<-EOT
Expand Down Expand Up @@ -369,7 +366,6 @@ module "eks_managed_node_group" {
]

ami_type = "BOTTLEROCKET_x86_64"
platform = "bottlerocket"

# this will get added to what AWS provides
bootstrap_extra_args = <<-EOT
Expand Down
10 changes: 10 additions & 0 deletions examples/eks_managed_node_group/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,16 @@ output "cluster_primary_security_group_id" {
value = module.eks.cluster_primary_security_group_id
}

output "cluster_service_cidr" {
description = "The CIDR block where Kubernetes pod and service IP addresses are assigned from"
value = module.eks.cluster_service_cidr
}

output "cluster_ip_family" {
description = "The IP family used by the cluster (e.g. `ipv4` or `ipv6`)"
value = module.eks.cluster_ip_family
}

################################################################################
# Access Entry
################################################################################
Expand Down
2 changes: 2 additions & 0 deletions examples/fargate_profile/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,12 +64,14 @@ No inputs.
| <a name="output_cluster_iam_role_unique_id"></a> [cluster\_iam\_role\_unique\_id](#output\_cluster\_iam\_role\_unique\_id) | Stable and unique string identifying the IAM role |
| <a name="output_cluster_id"></a> [cluster\_id](#output\_cluster\_id) | The ID of the EKS cluster. Note: currently a value is returned only for local EKS clusters created on Outposts |
| <a name="output_cluster_identity_providers"></a> [cluster\_identity\_providers](#output\_cluster\_identity\_providers) | Map of attribute maps for all EKS identity providers enabled |
| <a name="output_cluster_ip_family"></a> [cluster\_ip\_family](#output\_cluster\_ip\_family) | The IP family used by the cluster (e.g. `ipv4` or `ipv6`) |
| <a name="output_cluster_name"></a> [cluster\_name](#output\_cluster\_name) | The name of the EKS cluster |
| <a name="output_cluster_oidc_issuer_url"></a> [cluster\_oidc\_issuer\_url](#output\_cluster\_oidc\_issuer\_url) | The URL on the EKS cluster for the OpenID Connect identity provider |
| <a name="output_cluster_platform_version"></a> [cluster\_platform\_version](#output\_cluster\_platform\_version) | Platform version for the cluster |
| <a name="output_cluster_primary_security_group_id"></a> [cluster\_primary\_security\_group\_id](#output\_cluster\_primary\_security\_group\_id) | Cluster security group that was created by Amazon EKS for the cluster. Managed node groups use this security group for control-plane-to-data-plane communication. Referred to as 'Cluster security group' in the EKS console |
| <a name="output_cluster_security_group_arn"></a> [cluster\_security\_group\_arn](#output\_cluster\_security\_group\_arn) | Amazon Resource Name (ARN) of the cluster security group |
| <a name="output_cluster_security_group_id"></a> [cluster\_security\_group\_id](#output\_cluster\_security\_group\_id) | ID of the cluster security group |
| <a name="output_cluster_service_cidr"></a> [cluster\_service\_cidr](#output\_cluster\_service\_cidr) | The CIDR block where Kubernetes pod and service IP addresses are assigned from |
| <a name="output_cluster_status"></a> [cluster\_status](#output\_cluster\_status) | Status of the EKS cluster. One of `CREATING`, `ACTIVE`, `DELETING`, `FAILED` |
| <a name="output_cluster_tls_certificate_sha1_fingerprint"></a> [cluster\_tls\_certificate\_sha1\_fingerprint](#output\_cluster\_tls\_certificate\_sha1\_fingerprint) | The SHA1 fingerprint of the public key of the cluster's certificate |
| <a name="output_eks_managed_node_groups"></a> [eks\_managed\_node\_groups](#output\_eks\_managed\_node\_groups) | Map of attribute maps for all EKS managed node groups created |
Expand Down
10 changes: 10 additions & 0 deletions examples/fargate_profile/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,16 @@ output "cluster_primary_security_group_id" {
value = module.eks.cluster_primary_security_group_id
}

output "cluster_service_cidr" {
description = "The CIDR block where Kubernetes pod and service IP addresses are assigned from"
value = module.eks.cluster_service_cidr
}

output "cluster_ip_family" {
description = "The IP family used by the cluster (e.g. `ipv4` or `ipv6`)"
value = module.eks.cluster_ip_family
}

################################################################################
# Access Entry
################################################################################
Expand Down
Loading

0 comments on commit 74d3918

Please sign in to comment.