feat(runner): Replace patch by install ICU package for ARM runners (p…

…hilips-labs#1624) * Update arm-runner-patch.tpl The runtimeconfig.json files seem to have changed on the latest Amazon Linux AMI. When running the user data scripts, the patch of these files fails which causes initialization of the runner instance to fail. * fix patch again * install libicu60 instead of patching * remove arm-patch * Revert "remove arm-patch" This reverts commit 639c46e. * Add ARM64 documentation * Remove arm-runner-patch.tpl and include in install-runner.sh * add arm64, ephemeral and windows examples to github workflow
enverus-cts · Jan 18, 2022 · 74cfa51 · 74cfa51
1 parent a6e6efd
commit 74cfa51
Show file tree

Hide file tree

Showing 15 changed files with 248 additions and 62 deletions.
diff --git a/.github/workflows/terraform.yml b/.github/workflows/terraform.yml
@@ -43,7 +43,7 @@ jobs:
       fail-fast: false
       matrix:
         terraform: [0.14.3, 0.15.5, 1.0.8]
-        example: ["default", "ubuntu", "prebuilt"]
+        example: ["default", "ubuntu", "prebuilt", "arm64", "ephemeral", "windows"]
     defaults:
       run:
         working-directory: examples/${{ matrix.example }}

diff --git a/README.md b/README.md
@@ -93,7 +93,7 @@ To be able to support a number of use-cases the module has quite a lot configura
 
 #### ARM64 support via Graviton/Graviton2 instance-types
 
-When using the default example or top-level module, specifying an `instance_type` that matches a Graviton/Graviton 2 (ARM64) architecture (e.g. a1, t4g or any 6th-gen `g` or `gd` type), the sub-modules will be automatically configured to provision with ARM64 AMIs and leverage GitHub's ARM64 action runner. See below for more details.
+When using the default example or top-level module, specifying `instance_types` that match a Graviton/Graviton 2 (ARM64) architecture (e.g. a1, t4g or any 6th-gen `g` or `gd` type), you must also specify `runner_architecture = "arm64"` and the sub-modules will be automatically configured to provision with ARM64 AMIs and leverage GitHub's ARM64 action runner. See below for more details.
 
 ## Usages
 
@@ -184,8 +184,6 @@ module "github-runner" {
 }
 ```
 
-**ARM64** support: Specify an `a1`, `t4g` or `*6g*` (6th-gen Graviton2) instance type to stand up an ARM64 runner, otherwise the default is x86_64.
-
 Run terraform by using the following commands
 
 ```bash
@@ -322,9 +320,10 @@ This module also allows you to run agents from a prebuilt AMI to gain faster sta
 Examples are located in the [examples](./examples) directory. The following examples are provided:
 
 - _[Default](examples/default/README.md)_: The default example of the module
+- _[ARM64](examples/arm64/README.md)_: Example usage with ARM64 architecture
 - _[Ubuntu](examples/ubuntu/README.md)_: Example usage of creating a runner using Ubuntu AMIs.
 - _[Windows](examples/windows/README.md)_: Example usage of creating a runner using Windows as the OS.
-- _[Ephemeral](examples/ephemeral/README.md) : Example usages of ephemeral runners based on the default example.
+- _[Ephemeral](examples/ephemeral/README.md)_: Example usages of ephemeral runners based on the default example.
 - _[Prebuilt Images](examples/prebuilt/README.md)_: Example usages of deploying runners with a custom prebuilt image.
 - _[Permissions boundary](examples/permissions-boundary/README.md)_: Example usages of permissions boundaries.
 
@@ -345,7 +344,7 @@ The following sub modules are optional and are provided as example or utility:
 
 ### ARM64 configuration for submodules
 
-When using the top level module configure `runner_architecture = arm64` and insure the list of `instance_types` matches. When not using the top-level ensure the bot properties are set on the submodules. 
+When using the top level module configure `runner_architecture = "arm64"` and ensure the list of `instance_types` matches. When not using the top-level, ensure these properties are set on the submodules. 
 
 ## Debugging
 
@@ -354,7 +353,7 @@ In case the setup does not work as intended follow the trace of events:
 - In the GitHub App configuration, the Advanced page displays all webhook events that were sent.
 - In AWS CloudWatch, every lambda has a log group. Look at the logs of the `webhook` and `scale-up` lambdas.
 - In AWS SQS you can see messages available or in flight.
-- Once an EC2 instance is running, you can connect to it in the EC2 user interface using Session Manager. Check the user data script using `cat /var/log/user-data.log`. By default several log files of the instances are streamed to AWS CloudWatch, look for a log group named `<environment>/runners`. In the log group you should see at least the log streams for the user data installation and runner agent.
+- Once an EC2 instance is running, you can connect to it in the EC2 user interface using Session Manager (use `enable_ssm_on_runners = true`). Check the user data script using `cat /var/log/user-data.log`. By default several log files of the instances are streamed to AWS CloudWatch, look for a log group named `<environment>/runners`. In the log group you should see at least the log streams for the user data installation and runner agent.
 - Registered instances should show up in the Settings - Actions page of the repository or organization (depending on the installation mode).
 
 <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

diff --git a/examples/arm64/.terraform.lock.hcl b/examples/arm64/.terraform.lock.hcl
diff --git a/examples/arm64/README.md b/examples/arm64/README.md
@@ -0,0 +1,31 @@
+# Action runners deployment with ARM64 architecture
+
+This module shows how to create GitHub action runners using AWS Graviton instances which have ARM64 architecture. Lambda release will be downloaded from GitHub.
+
+## Usages
+
+Steps for the full setup, such as creating a GitHub app can be found in the root module's [README](../../README.md). First download the Lambda releases from GitHub. Alternatively you can build the lambdas locally with Node or Docker, there is a simple build script in `<root>/.ci/build.sh`. In the `main.tf` you can simply remove the location of the lambda zip files, the default location will work in this case.
+
+> Ensure you have set the version in `lambdas-download/main.tf` for running the example. The version needs to be set to a GitHub release version, see https://github.com/philips-labs/terraform-aws-github-runner/releases
+
+```bash
+cd lambdas-download
+terraform init
+terraform apply
+cd ..
+```
+
+Before running Terraform, ensure the GitHub app is configured. See the [configuration details](../../README.md#usages) for more details.
+
+```bash
+terraform init
+terraform apply
+```
+
+You can receive the webhook details by running:
+
+```bash
+terraform output -raw webhook_secret
+```
+
+Be-aware some shells will print some end of line character `%`. 
diff --git a/examples/arm64/lambdas-download/main.tf b/examples/arm64/lambdas-download/main.tf
@@ -0,0 +1,25 @@
+locals {
+  version = "<REPLACE_BY_GITHUB_RELEASE_VERSION>"
+}
+
+module "lambdas" {
+  source = "../../../modules/download-lambda"
+  lambdas = [
+    {
+      name = "webhook"
+      tag  = local.version
+    },
+    {
+      name = "runners"
+      tag  = local.version
+    },
+    {
+      name = "runner-binaries-syncer"
+      tag  = local.version
+    }
+  ]
+}
+
+output "files" {
+  value = module.lambdas.files
+}
diff --git a/examples/arm64/main.tf b/examples/arm64/main.tf
@@ -0,0 +1,77 @@
+locals {
+  environment = "default"
+  aws_region  = "eu-west-1"
+}
+
+resource "random_id" "random" {
+  byte_length = 20
+}
+
+
+################################################################################
+### Hybrid account
+################################################################################
+
+module "runners" {
+  source                          = "../../"
+  create_service_linked_role_spot = true
+  aws_region                      = local.aws_region
+  vpc_id                          = module.vpc.vpc_id
+  subnet_ids                      = module.vpc.private_subnets
+
+  environment = local.environment
+  tags = {
+    Project = "ProjectX"
+  }
+
+  github_app = {
+    key_base64     = var.github_app_key_base64
+    id             = var.github_app_id
+    webhook_secret = random_id.random.hex
+  }
+
+  # Grab zip files via lambda_download, will automatically get the ARM64 build
+  webhook_lambda_zip                = "lambdas-download/webhook.zip"
+  runner_binaries_syncer_lambda_zip = "lambdas-download/runner-binaries-syncer.zip"
+  runners_lambda_zip                = "lambdas-download/runners.zip"
+
+  enable_organization_runners = false
+  # Runners will automatically get the "arm64" label
+  runner_extra_labels         = "default,example"
+
+  # enable access to the runners via SSM
+  enable_ssm_on_runners = true
+
+  # use S3 or KMS SSE to runners S3 bucket
+  # runner_binaries_s3_sse_configuration = {
+  #   rule = {
+  #     apply_server_side_encryption_by_default = {
+  #       sse_algorithm = "AES256"
+  #     }
+  #   }
+  # }
+
+  # Uncommet idle config to have idle runners from 9 to 5 in time zone Amsterdam
+  # idle_config = [{
+  #   cron      = "* * 9-17 * * *"
+  #   timeZone  = "Europe/Amsterdam"
+  #   idleCount = 1
+  # }]
+
+  # Let the module manage the service linked role
+  # create_service_linked_role_spot = true
+
+  runner_architecture = "arm64"
+  # Ensure all instance types have ARM64 architecture (ie. AWS Graviton processors)
+  instance_types = ["t4g.large", "c6g.large"]
+
+  # override delay of events in seconds
+  delay_webhook_event   = 5
+  runners_maximum_count = 1
+
+  # set up a fifo queue to remain order
+  fifo_build_queue = true
+
+  # override scaling down
+  scale_down_schedule_expression = "cron(* * * * ? *)"
+}
diff --git a/examples/arm64/outputs.tf b/examples/arm64/outputs.tf
@@ -0,0 +1,15 @@
+output "runners" {
+  value = {
+    lambda_syncer_name = module.runners.binaries_syncer.lambda.function_name
+  }
+}
+
+output "webhook_endpoint" {
+  value = module.runners.webhook.endpoint
+}
+
+output "webhook_secret" {
+  sensitive = true
+  value     = random_id.random.hex
+}
+
diff --git a/examples/arm64/providers.tf b/examples/arm64/providers.tf
@@ -0,0 +1,3 @@
+provider "aws" {
+  region = local.aws_region
+}
diff --git a/examples/arm64/variables.tf b/examples/arm64/variables.tf
@@ -0,0 +1,4 @@
+
+variable "github_app_key_base64" {}
+
+variable "github_app_id" {}
diff --git a/examples/arm64/versions.tf b/examples/arm64/versions.tf
@@ -0,0 +1,15 @@
+terraform {
+  required_providers {
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 3.27"
+    }
+    local = {
+      source = "hashicorp/local"
+    }
+    random = {
+      source = "hashicorp/random"
+    }
+  }
+  required_version = ">= 0.14"
+}
diff --git a/examples/arm64/vpc.tf b/examples/arm64/vpc.tf
@@ -0,0 +1,7 @@
+module "vpc" {
+  source = "git::https://github.com/philips-software/terraform-aws-vpc.git?ref=2.2.0"
+
+  environment                = local.environment
+  aws_region                 = local.aws_region
+  create_private_hosted_zone = false
+}
diff --git a/examples/default/main.tf b/examples/default/main.tf
@@ -9,7 +9,7 @@ resource "random_id" "random" {
 
 
 ################################################################################
-### Hybrid acccount
+### Hybrid account
 ################################################################################
 
 module "runners" {

diff --git a/modules/runners/main.tf b/modules/runners/main.tf
@@ -12,7 +12,6 @@ locals {
   instance_profile_path = var.instance_profile_path == null ? "/${var.environment}/" : var.instance_profile_path
   lambda_zip            = var.lambda_zip == null ? "${path.module}/lambdas/runners/runners.zip" : var.lambda_zip
   userdata_template     = var.userdata_template == null ? local.default_userdata_template[var.runner_os] : var.userdata_template
-  userdata_arm_patch    = "${path.module}/templates/arm-runner-patch.tpl"
   kms_key_arn           = var.kms_key_arn != null ? var.kms_key_arn : ""
 
   default_ami = {
@@ -119,7 +118,7 @@ resource "aws_launch_template" "runner" {
     pre_install = var.userdata_pre_install
     install_runner = templatefile(local.userdata_install_runner[var.runner_os], {
       S3_LOCATION_RUNNER_DISTRIBUTION = var.s3_location_runner_binaries
-      ARM_PATCH                       = var.runner_architecture == "arm64" ? templatefile(local.userdata_arm_patch, {}) : ""
+      RUNNER_ARCHITECTURE             = var.runner_architecture
     })
     post_install    = var.userdata_post_install
     start_runner    = templatefile(local.userdata_start_runner[var.runner_os], {})

diff --git a/modules/runners/templates/arm-runner-patch.tpl b/modules/runners/templates/arm-runner-patch.tpl
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,4 @@

		variable "github_app_key_base64" {}

		variable "github_app_id" {}