EKS Terraform Quickstart template

Use this template to easily create a new Git Repository for managing Jenkins X cloud infrastructure needs.

We recommend using Terraform to manage the infrastructure needed to run Jenkins X. There can be a number of cloud resources which need to be created such as:

• Kubernetes cluster
• Storage buckets for long term storage of logs
• IAM Bindings to manage permissions for applications using cloud resources

Jenkins X likes to use GitOps to manage the lifecycle of both infrastructure and cluster resources.
This requires two git Repositories to achieve this:

• the first, infrastructure resources will be managed by Terraform and will keep resourecs in sync.
• the second, the Kubernetes specific cluster resources will be managed by Jenkins X and keep resources in sync.

Prerequisites

• A Git organisation that will be used to create the GitOps repositories used for Jenkins X below. e.g. https://github.com/organizations/plan.

• Create a git bot user (different from your own personal user) e.g. https://github.com/join and generate a a personal access token, this will be used by Jenkins X to interact with git repositories. e.g. https://github.com/settings/tokens/new?scopes=repo,read:user,read:org,user:email,write:repo_hook,delete_repo,admin:repo_hook

• This bot user needs to have write permission to write to any git repository used by Jenkins X. This can be done by adding the bot user to the git organisation level or individual repositories as a collaborator Add the new bot user to your Git Organisation, for now give it Owner permissions, we will reduce this to member permissions soon.

• Install terraform CLI - see here

• Install jx CLI - see here

• AWS account and credentials

Git repositories

We use 2 git repositories:

• Infrastructure git repository for the Terraform configuration to setup/upgrade/modify your cloud infrastructure (kubernetes cluster, IAM accounts, IAM roles, buckets etc)
• Cluster git repository to contain the helmfile.yaml file to define the helm charts to deploy in your cluster

We use separate git repositories since the infrastructure tends to change rarely; whereas the cluster git repository changes alot (every time you add a new quickstart, import a project, release a project etc).

Often different teams look after infrastructure; or you may use tools like Terraform Cloud to process changes to infrastructure & review changes to infrastructure more closely than promotion of applications.

Getting started

Note: remember to create the Git repositories below in your Git Organisation rather than your personal Git account else this will lead to issues with ChatOps and automated registering of webhooks

1. Create and clone your Infrastructure git repository from this GitHub Template https://github.com/jx3-gitops-repositories/jx3-terraform-eks/generate
2. Create a Cluster git repository from this template https://github.com/jx3-gitops-repositories/jx3-eks-vault/generate
3. Override the variable defaults in the Infrastructure repository. (E.g, edit variables.tf, set TF_VAR_ environment variables, or pass the values on the terraform command line.)

• cluster_version: Kubernetes version for the EKS cluster.
• region: AWS region code for the AWS region to create the cluster in.
• jx_git_url: URL of the Cluster repository.
• jx_bot_username: The username of the git bot user

4. (Optional) Depending on how you want to authenticate with AWS you might want to adjust the AWS provider block in main.tf according to this documentation.
5. commit and push any changes to your Infrastructure git repository:

git commit -a -m "fix: configure cluster repository and project"
git push

6. Define an environment variable to pass the bot token into Terraform:

export TF_VAR_jx_bot_token=my-bot-token

7. Now, initialise, plan and apply Terraform:

terraform init

terraform plan

terraform apply

Tail the Jenkins X installation logs

$(terraform output follow_install_logs)

Once finished you can now move into the Jenkins X Developer namespace

jx ns jx

and create or import your applications

jx project

If your application is not yet in a git repository, you are asked for a github.com user and token to push the application to git. This user needs administrative permissions to create repository and hooks. It is likely not the same user as the bot user mentioned above.

Terraform

For the full list of terraform inputs in the eks-jx module se see the documentation for jenkins-x/terraform-aws-eks-jx

Providers

Name	Version
aws	5.60.0
random	3.6.2

Modules

Name	Source	Version
eks	terraform-aws-modules/eks/aws	20.20.0
eks-jx	github.com/jenkins-x/terraform-aws-eks-jx	v3.0.1
iam-assumable-role-ebs-csi	terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc	~> v3.8.0
vpc	terraform-aws-modules/vpc/aws	5.9.0

Inputs

Name	Description	Type	Default	Required
ami_type	ami type for the node group worker intances	string	"AL2023_x86_64_STANDARD"	no
cluster_endpoint_private_access	Indicates whether or not the Amazon EKS private API server endpoint is enabled.	bool	false	no
cluster_endpoint_public_access	Indicates whether or not the Amazon EKS public API server endpoint is enabled.	bool	true	no
cluster_in_private_subnet	Flag to enable installation of cluster on private subnets	bool	true	no
cluster_name	Name of the Kubernetes cluster to create	string	""	no
cluster_version	Kubernetes version to use for the EKS cluster.	string	n/a	yes
enable_nat_gateway	n/a	bool	true	no
force_destroy	Flag to determine whether storage buckets get forcefully destroyed. If set to false, empty the bucket first in the aws s3 console, else terraform destroy will fail with BucketNotEmpty error	bool	false	no
jx_bot_token	Bot token used to interact with the Jenkins X cluster git repository	string	n/a	yes
jx_bot_username	Bot username used to interact with the Jenkins X cluster git repository	string	n/a	yes
jx_git_url	URL for the Jenins X cluster git repository	string	n/a	yes
nginx_chart_version	nginx chart version	string	"3.12.0"	no
node_machine_type	n/a	string	"m6i.large"	no
private_subnets	n/a	list(string)	[ "10.0.4.0/24", "10.0.5.0/24", "10.0.6.0/24" ]	no
public_subnets	n/a	list(string)	[ "10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24" ]	no
region	AWS region code for creating resources.	string	n/a	yes
single_nat_gateway	n/a	bool	true	no
use_asm	Flag to specify if resources for AWS Secret MAanger should be created instead of vault resources	bool	false	no
vpc_cidr_block	n/a	string	"10.0.0.0/16"	no
vpc_name	VPC	string	"tf-vpc-eks"	no

Outputs

Name	Description
cert_manager_iam_role	The IAM Role that the Cert Manager pod will assume to authenticate
cluster_autoscaler_iam_role	The IAM Role that the Jenkins X UI pod will assume to authenticate
cluster_name	The name of the created cluster
cluster_oidc_issuer_url	The Cluster OIDC Issuer URL
cm_cainjector_iam_role	The IAM Role that the CM CA Injector pod will assume to authenticate
controllerbuild_iam_role	The IAM Role that the ControllerBuild pod will assume to authenticate
docs	Follow Jenkins X 3.x alpha docs for more information
external_dns_iam_role	The IAM Role that the External DNS pod will assume to authenticate
follow_install_logs	Follow Jenkins X install logs
lts_logs_bucket	The bucket where logs from builds will be stored
lts_reports_bucket	The bucket where test reports will be stored
lts_repository_bucket	The bucket that will serve as artifacts repository
tekton_bot_iam_role	The IAM Role that the build pods will assume to authenticate

Cleanup

To remove any cloud resources created here:

• Manually remove the generated load balancer, for example, through the AWS EC2 console "Load Balancers" tab. The load balancer is currently not cleaned up automatically and may cause the following destroy step to hang and finally fail.
• Run:

terraform destroy

Contributing

When adding new variables please regenerate the markdown table

terraform-docs markdown table .

and replace the Inputs section above

Formatting

When developing please remember to format codebase before raising a pull request

terraform fmt -check -diff -recursive