-
Notifications
You must be signed in to change notification settings - Fork 922
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
1c26c8f
commit 848d9ae
Showing
10 changed files
with
507 additions
and
7 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
138 changes: 138 additions & 0 deletions
138
blueprints/third-party-solutions/gitlab-runner/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,138 @@ | ||
| # GitLab Runner Blueprint | ||
|
|
||
| This Terraform module provides a streamlined and automated way to provision and | ||
| configure a GitLab CI/CD runner on Google Cloud Platform (GCP). It leverages | ||
| GCP's infrastructure and GitLab's runner capabilities to create a flexible and | ||
| scalable solution for your CI/CD pipelines. | ||
|
|
||
| ## Gitlab Runner options | ||
|
|
||
| ### Docker executor | ||
|
|
||
| In GitLab CI/CD, | ||
| the ["docker executor"](https://docs.gitlab.com/runner/executors/docker.html) is | ||
| one of the methods used to determine | ||
| the environment in which your CI/CD jobs will be executed. When you use the | ||
| docker executor, your jobs run within Docker containers. | ||
|
|
||
|  | ||
|
|
||
| Key features and how it works: | ||
|
|
||
| - **Leverages Docker**: The GitLab Runner relies on the presence of Docker on | ||
| the machine (or host) where it is installed. During job execution, the runner | ||
| creates and manages Docker containers to run the steps defined in your | ||
| .gitlab-ci.yml file. | ||
|
|
||
| - **Job Isolation**: Each job is executed within its own, isolated Docker | ||
| container. This provides a clean and independent environment for each build or | ||
| test process, preventing conflicts or side effects from previous jobs. | ||
|
|
||
| - **Flexibility through Images**: | ||
| You specify the Docker image to be used as the base for each container. This | ||
| allows you to choose the operating system, tools, and pre-installed software | ||
| needed for your build environment. You can use official images from Docker Hub | ||
| or create your custom images. | ||
|
|
||
| Please find below a sample terraform.tfvars for bootstrapping a Gitlab runner | ||
| with Docker executor via this blueprint. | ||
|
|
||
| ```tfvars | ||
| project_id = "test-project" | ||
| gitlab_config = { | ||
| hostname = "gitlab.gcp.example.com" | ||
| ca_cert_pem = "-----BEGIN CERTIFICATE-----.." | ||
| } | ||
| gitlab_runner_config = { | ||
| authentication_token = "auth-token" | ||
| executors_config = { | ||
| docker = {} | ||
| } | ||
| } | ||
| network_config = { | ||
| network_self_link = "https://www.googleapis.com/compute/v1/projects/test-net-project/global/networks/default" | ||
| subnet_self_link = "https://www.googleapis.com/compute/v1/projects/test-net-project/regions/europe-west1/subnetworks/subnet" | ||
| } | ||
| # tftest skip | ||
| ``` | ||
|
|
||
| ### Docker Autoscale Executor (BETA) | ||
|
|
||
| The ["docker autoscaler executor"](https://docs.gitlab.com/runner/executors/docker_autoscaler.html) | ||
| in GitLab is a powerful mechanism designed to dynamically scale your CI/CD | ||
| resources based on the demand of your pipelines. It achieves this by | ||
| intelligently managing virtual machines (VMs) on cloud platforms, provisioning | ||
| or terminating VMs as needed. | ||
|
|
||
|  | ||
|
|
||
| Key Features: | ||
|
|
||
| - **Dynamic Scaling**: When the workload increases (many CI/CD jobs queued), the | ||
| docker autoscaler executor instructs a cloud provider to spin up new VMs to | ||
| handle the jobs. Conversely, when jobs are completed and the workload | ||
| decreases, it terminates VMs to release resources. | ||
|
|
||
| - **Cost Optimization**: By matching resource allocation to your CI/CD demands, | ||
| you avoid paying for idle VMs when they are not needed. | ||
|
|
||
| - **Flexibility**: The docker autoscaler executor wraps the docker executor, | ||
| meaning you still benefit from running jobs in isolated Docker containers. | ||
|
|
||
| The docker autoscaler executor achieves its dynamic scaling capabilities through | ||
| integration | ||
| with [Fleeting plugins](https://gitlab.com/gitlab-org/fleeting/fleeting-plugin-googlecompute). | ||
| Fleeting is a framework that abstracts the management of autoscaled VM | ||
| instances, and it offers plugins for different cloud providers, including Google | ||
| Cloud Platform (GCP). | ||
|
|
||
| Please find below a sample terraform.tfvars for bootstrapping a Gitlab runner | ||
| with Docker executor via this blueprint. | ||
|
|
||
| ```tfvars | ||
| project_id = "test-project" | ||
| gitlab_config = { | ||
| hostname = "gitlab.gcp.example.com" | ||
| ca_cert_pem = "-----BEGIN CERTIFICATE-----.." | ||
| } | ||
| gitlab_runner_config = { | ||
| authentication_token = "auth-token" | ||
| executors_config = { | ||
| docker_autoscaler = { | ||
| gcp_project_id = "test-project | ||
| zone = "europe-west1-b" | ||
| mig_name = "gitlab-runner" | ||
| machine_type = "g1-small" | ||
| machine_image = "coreos-cloud/global/images/family/coreos-stable" | ||
| network_tags = ["gitlab-runner"] | ||
| } | ||
| } | ||
| } | ||
| network_config = { | ||
| network_self_link = "https://www.googleapis.com/compute/v1/projects/test-net-project/global/networks/default" | ||
| subnet_self_link = "https://www.googleapis.com/compute/v1/projects/test-net-project/regions/europe-west1/subnetworks/subnet" | ||
| } | ||
| # tftest skip | ||
| ``` | ||
|
|
||
| #### Requirements | ||
|
|
||
| Please be aware of the following requirements for the Docker Autoscaler executor Gitlab runner to work properly in Google Cloud Platform: | ||
|
|
||
| - Ensure the gitlab runner (Manager) can connect to the Compute Engine VMs being part of the Managed Instance Group (running the docker executor) on port 22 | ||
|
|
||
| - Do not enforce os-login either at project or instance level otherwise the Gitlab Runner manager won't be able to connect to the instances with SSH keys added on the Compute Engine metadata. More information on this in the GCP documentation available at the following [link](https://cloud.google.com/compute/docs/troubleshooting/troubleshooting-ssh-errors#linux_errors). | ||
| <!-- BEGIN TFDOC --> | ||
| ## Variables | ||
|
|
||
| | name | description | type | required | default | | ||
| |---|---|:---:|:---:|:---:| | ||
| | [gitlab_config](variables.tf#L23) | Gitlab server configuration. | <code title="object({ hostname = optional(string, "gitlab.gcp.example.com") ca_cert_pem = optional(string, null) })">object({…})</code> | ✓ | | | ||
| | [gitlab_runner_config](variables.tf#L31) | Gitlab Runner config. | <code title="object({ authentication_token = string executors_config = object({ docker_autoscaler = optional(object({ gcp_project_id = string zone = optional(string, "europe-west1-b") mig_name = optional(string, "gitlab-runner") machine_type = optional(string, "g1-small") machine_image = optional(string, "coreos-cloud/global/images/family/coreos-stable") network_tags = optional(list(string), ["gitlab-runner"]) }), null) docker = optional(object({ tls_verify = optional(bool, true) }), null) }) })">object({…})</code> | ✓ | | | ||
| | [network_config](variables.tf#L58) | Shared VPC network configurations to use for Gitlab Runner VM. | <code title="object({ host_project = optional(string) network_self_link = string subnet_self_link = string })">object({…})</code> | ✓ | | | ||
| | [prefix](variables.tf#L67) | Prefix used for resource names. | <code>string</code> | ✓ | | | ||
| | [project_id](variables.tf#L86) | Project id, references existing project if `project_create` is null. | <code>string</code> | ✓ | | | ||
| | [vm_config](variables.tf#L91) | Gitlab runner GCE config. | <code title="object({ boot_disk_size = optional(number, 100) name = optional(string, "gitlab-runner-0") instance_type = optional(string, "e2-standard-2") network_tags = optional(list(string), []) zone = optional(string, "europe-west1-b") })">object({…})</code> | ✓ | | | ||
| | [admin_principals](variables.tf#L17) | Users, groups and/or service accounts that are assigned roles, in IAM format (`group:[email protected]`). | <code>list(string)</code> | | <code>[]</code> | | ||
| | [project_create](variables.tf#L77) | Provide values if project creation is needed, uses existing project if null. Parent is in 'folders/nnn' or 'organizations/nnn' format. | <code title="object({ billing_account_id = string parent = string })">object({…})</code> | | <code>null</code> | | ||
| <!-- END TFDOC --> |
33 changes: 33 additions & 0 deletions
33
...rints/third-party-solutions/gitlab-runner/assets/config/docker_autoscaler_config.toml.tpl
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| [[runners]] | ||
| url = "GITLAB_CI_URI" | ||
| name = "RUNNER_NAME" | ||
| token = "RUNNER_TOKEN" | ||
| shell = "sh" # use powershell or pwsh for Windows Images | ||
|
|
||
| executor = "docker-autoscaler" | ||
|
|
||
| # Docker Executor config | ||
| [runners.docker] | ||
| image = "alpine:latest" | ||
|
|
||
| # Autoscaler config | ||
| [runners.autoscaler] | ||
| plugin = "fleeting-plugin-googlecompute" | ||
|
|
||
| capacity_per_instance = 5 | ||
| max_use_count = 1 | ||
| max_instances = 10 | ||
|
|
||
| [runners.autoscaler.plugin_config] # plugin specific configuration (see plugin documentation) | ||
| name = "${mig_name}" # GCP Instance Group name | ||
| project = "${gcp_project_id}" | ||
| zone = "${zone}" | ||
| # credentials_file = "/home/user/.config/gcloud/application_default_credentials.json" # optional, default is '~/.config/gcloud/application_default_credentials.json' | ||
|
|
||
| [runners.autoscaler.connector_config] | ||
| username = "runner" | ||
| use_external_addr = false | ||
|
|
||
| [[runners.autoscaler.policy]] | ||
| idle_count = 5 | ||
| idle_time = "20m0s" |
15 changes: 15 additions & 0 deletions
15
blueprints/third-party-solutions/gitlab-runner/assets/config/docker_config.toml.tpl
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| [[runners]] | ||
| url = "GITLAB_CI_URI" | ||
| name = "RUNNER_NAME" | ||
| token = "RUNNER_TOKEN" | ||
| executor = "docker" | ||
| [runners.docker] | ||
| tls_verify = true | ||
| image = "alpine:latest" | ||
| privileged = false | ||
| disable_entrypoint_overwrite = false | ||
| oom_kill_disable = false | ||
| disable_cache = false | ||
| volumes = ["/cache"] | ||
| shm_size = 0 | ||
| network_mtu = 0 |
61 changes: 61 additions & 0 deletions
61
blueprints/third-party-solutions/gitlab-runner/assets/startup-script.sh.tpl
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # Copyright 2024 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| #!/bin/bash | ||
|
|
||
| GITLAB_URL=https://${gitlab_hostname} | ||
| GITLAB_RUNNER_CONFIG=${gitlab_runner_config} | ||
|
|
||
| GL_NAME=$(curl 169.254.169.254/computeMetadata/v1/instance/name --header "Metadata-Flavor:Google") | ||
| GL_EXECUTOR=$(curl 169.254.169.254/computeMetadata/v1/instance/attributes/gl_executor --header "Metadata-Flavor:Google") | ||
|
|
||
| apt update | ||
| curl --location "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | bash | ||
|
|
||
| # Fetch Gitlab Server SSL Private and public keys | ||
| echo "${gitlab_ca_cert}" | base64 -d -w0 >/tmp/ca.crt | ||
| cp -f /tmp/ca.crt /usr/local/share/ca-certificates/ | ||
| update-ca-certificates | ||
|
|
||
| # Install Docker | ||
| # Add Docker's official GPG key: | ||
| apt-get update | ||
| apt-get install -yq ca-certificates curl gnupg | ||
| install -m 0755 -d /etc/apt/keyrings | ||
| curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg | ||
| chmod a+r /etc/apt/keyrings/docker.gpg | ||
|
|
||
| # Add the repository to Apt sources: | ||
| echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | tee /etc/apt/sources.list.d/docker.list >/dev/null | ||
| apt-get update | ||
| apt-get install -yq docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin | ||
|
|
||
| # setup new gitlab runner config | ||
| echo $GITLAB_RUNNER_CONFIG | base64 -d > /etc/gitlab-runner/config.toml | ||
|
|
||
| # Install Gitlab Runner | ||
| apt install -y gitlab-runner | ||
|
|
||
| %{ if gitlab_executor_type == "docker-autoscaler" } | ||
| # Install GCP fleeting plugin for Docker Autoscale Runner | ||
| # https://docs.gitlab.com/runner/executors/docker_autoscaler.html#install-a-fleeting-plugin | ||
| wget https://gitlab.com/gitlab-org/fleeting/fleeting-plugin-googlecompute/-/releases/v0.1.0/downloads/fleeting-plugin-googlecompute-linux-386 | ||
| chmod +x fleeting-plugin-googlecompute-linux-386 | ||
| mv ./fleeting-plugin-googlecompute-linux-386 /usr/bin/fleeting-plugin-googlecompute | ||
| %{ endif } | ||
|
|
||
| gitlab-runner register --non-interactive --name="$GL_NAME" \ | ||
| --url="$GITLAB_URL" --token="${token}" --template-config="/etc/gitlab-runner/config.toml" | ||
| --executor="${gitlab_executor_type}" | ||
| systemctl restart gitlab-runner |
Binary file added
BIN
+147 KB
blueprints/third-party-solutions/gitlab-runner/images/docker-autoscaler.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| /** | ||
| * Copyright 2024 Google LLC | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| locals { | ||
| role_id = "projects/${module.project.project_id}/roles/${local.role_name}" | ||
| role_name = "gitlab_runner_manager_role" | ||
| runner_config_type = [for key, value in var.gitlab_runner_config.executors_config : key if value != null][0] | ||
| runner_startup_script_config = { | ||
| gitlab_hostname = var.gitlab_config.hostname | ||
| gitlab_ca_cert = base64encode(var.gitlab_config.ca_cert_pem) | ||
| token = var.gitlab_runner_config.authentication_token | ||
| gitlab_runner_config = base64encode(templatefile("${path.module}/assets/config/${local.runner_config_type}_config.toml.tpl", var.gitlab_runner_config.executors_config[local.runner_config_type])) | ||
| gitlab_executor_type = replace(local.runner_config_type, "_", "-") | ||
| } | ||
| } | ||
|
|
||
| resource "google_service_account_iam_member" "admin-account-iam" { | ||
| count = local.runner_config_type == "docker_autoscaler" ? 1 : 0 | ||
| service_account_id = module.gitlab-runner-template.0.service_account.name | ||
| role = "roles/iam.serviceAccountUser" | ||
| member = "serviceAccount:${module.gitlab-runner.service_account.email}" | ||
| } | ||
|
|
||
| module "project" { | ||
| source = "../../../modules/project" | ||
| parent = try(var.project_create.parent, null) | ||
| billing_account = try(var.project_create.billing_account_id, null) | ||
| prefix = var.project_create == null ? null : var.prefix | ||
| name = var.project_id | ||
| project_create = var.project_create != null | ||
| custom_roles = { | ||
| (local.role_name) = [ | ||
| "compute.instanceGroupManagers.get", | ||
| "compute.instanceGroupManagers.update", | ||
| "compute.instances.get", | ||
| "compute.instances.setMetadata" | ||
| ] | ||
| } | ||
| iam = { | ||
| (local.role_id) = ["serviceAccount:${module.gitlab-runner.service_account.email}"] | ||
| } | ||
| services = [ | ||
| "compute.googleapis.com", | ||
| "storage.googleapis.com", | ||
| "stackdriver.googleapis.com", | ||
| "dns.googleapis.com", | ||
| "iam.googleapis.com", | ||
| ] | ||
| shared_vpc_service_config = { | ||
| attach = true | ||
| host_project = var.network_config.host_project | ||
| service_identity_iam = { | ||
| "roles/compute.networkUser" = [ | ||
| "cloudservices", "compute" | ||
| ] | ||
| } | ||
| network_users = var.admin_principals | ||
| } | ||
| } |
Oops, something went wrong.