Skip to content

This module provides an easy way to deploy GitLab Runners on Kubernetes using Terraform using official Helm chart..

Notifications You must be signed in to change notification settings

MaciekLeks/lazy-gitlab-runner-k8s-tf

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

59 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Lazy GitLab Runner Kubernetes Terraform Module

This module provides an easy way to deploy GitLab Runners on Kubernetes using Terraform. It is designed to replace the functionality of terraform-kubernetes-gitlab-runner while addressing some key limitations.

Why a New Module?

GitLab has been systematically moving configuration attributes to TOML format. To accommodate this shift and provide a more flexible solution, this new module leverages the Tobotimus/toml provider. This allows for easier management of TOML configurations within Terraform.

Why "Lazy"?

We call this module "lazy" because it uses a simple approach, unlike the more complex solutions in the old project. Here's why:

  1. Easy Setup: We keep things consistent between Terraform and GitLab. For example, if GitLab uses camelCase, we use it in Terraform too. This:

    • Makes things simpler
    • Matches GitLab's own instructions better
    • Helps users switch between GitLab's docs and our module easily
  2. Straightforward Design: What you put into Terraform is directly reflected in the GitLab Runner setup. This makes it easier to understand and fix if needed.

  3. Clear Structure: I've removed unnecessary complications, making the module easier to understand and fix problems.

This "lazy" way focuses on keeping things simple and clear. It makes the module easier to use, especially for people who already know how to set up GitLab Runners.

Key Features

  • Simplified deployment of GitLab Runners on Kubernetes
  • Utilizes the Tobotimus/toml provider for TOML encoding
  • Designed to be more adaptable to GitLab's evolving configuration standards

Getting Started

The first stable version of this module is now available (since v0.2.0)! You can find example configurations in the samples directory.

Development Status

  • Core functionality implementation
  • Testing and validation
  • Documentation
  • Example configurations
  • First stable release

We appreciate your interest in this project. If you'd like to contribute or stay updated on its progress, please:

  1. Star this repository to show your support
  2. Watch this repository for updates
  3. Check the Issues tab for current development tasks and known issues

Contributing

Contributions are welcome! Just write an issue and create a pull request.

Terraform Documentation

Requirements

Name Version
terraform >= 1.8.0
helm >= 2.15.0
kubernetes >= 2.23.0
toml >= 0.3.0

Providers

Name Version
helm 2.15.0

Modules

No modules.

Resources

Name Type
helm_release.gitlab_runner resource

Inputs

Name Description Type Default Required
affinity Affinity for runner pod assignment.
object({
nodeAffinity : optional(object({
preferredDuringSchedulingIgnoredDuringExecution : optional(list(object({
weight : number
preference : object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchFields : optional(list(object({
key : string
operator : string
values : list(string)
})))
})
})), null)
requiredDuringSchedulingIgnoredDuringExecution : optional(list(object({
nodeSelectorTerms : object({
matchExpressions : optional(object({
key : string
operator : string
values : list(string)
}))
matchFields : optional(object({
key : string
operator : string
values : list(string)
}))
})
})), null)
}), null)

podAffinity : optional(object({
preferredDuringSchedulingIgnoredDuringExecution : optional(list(object({
podAffinityTerm : object({
weight : number
topology_key : string
namespaces : optional(list(string))
labelSelector : optional(object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchLabels : optional(list(string))
}))
namespaceSelector : optional(object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchLabels : optional(list(string))
}))
})
})), null)
requiredDuringSchedulingIgnoredDuringExecution : optional(list(object({
topology_key : string
namespaces : optional(list(string))
labelSelector : optional(object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchLabels : optional(list(string))
}))
namespaceSelector : optional(object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchLabels : optional(list(string))
}))
})), null)
}), null)

podAntiAffinity : optional(object({
preferredDuringSchedulingIgnoredDuringExecution : optional(list(object({
podAffinityTerm : object({
weight : number
topology_key : string
namespaces : optional(list(string))
labelSelector : optional(object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchLabels : optional(list(string))
}))
namespaceSelector : optional(object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchLabels : optional(list(string))
}))
})
})), null)
requiredDuringSchedulingIgnoredDuringExecution : optional(list(object({
topology_key : string
namespaces : optional(list(string))
labelSelector : optional(object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchLabels : optional(list(string))
}))
namespaceSelector : optional(object({
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
matchLabels : optional(list(string))
}))
})), null)
}), null)
})
{} no
automountServiceAccountToken Automount service account token in the deployment.. bool false no
certSecretName Set the certsSecretName in order to pass custom certficates for GitLab Runner to use. string null no
checkInterval Defines in seconds how often to check GitLab for a new builds. number 3 no
concurrent Configure the maximum number of concurrent jobs number 10 no
configMaps Additional ConfiMaps to be mounted. map(any) null no
connectionMaxAge Configure GitLab Runner's maximum connection age for TLS keepalive connections. string "15m0s" no
deploymentAnnotations Annotations to be added to the runner deployment. map(string) {} no
deploymentLabels Labels to be added to the runner deployment. map(string) {} no
deploymentLifecycle Configure the lifecycle of the runner deployment. map(any) {} no
envVars Configure environment variables that will be present when the registration command runs.
list(object({
name : string
value : string
}))
null no
extraEnv Extra environment variables to be added to the runner pods. map(string) {} no
extraEnvFrom Additional environment variables from other data sources (k8s secrets).
map(object({
secretKeyRef : object({
name : string
key : string
})
}))
{} no
extraObjects Additional k8s objects to be created. list(map(any)) [] no
fullnameOverride Override the full name of the k8s resources. string null no
gitlabUrl The GitLab Server URL (with protocol) that want to register the runner against. string n/a yes
helm_settings The settings for the Helm chart.
object({
name = optional(string, "gitlab-runner")
repository = optional(string, "https://charts.gitlab.io")
chart = optional(string, "gitlab-runner")
namespace = optional(string, "gitlab-runner")
version = optional(string, null) //default to last version
create_namespace = optional(bool, false)
atomic = optional(bool, true)
wait = optional(bool, true)
timeout = optional(number, 300)
})
{} no
hostAliases List of hosts and IPs that will be injected into the pod's hosts file.
list(object({
ip : string
hostnames : list(string)
}))
[] no
hpa Horizontal Pod Autoscaling with API limited to metrics specification only (api/version: autoscaling/v2).
object({
minReplicas = number
maxReplicas = number
behavior = object({
scale_up = object({
stabilizationWindowSeconds = number
selectPolicy = string
policies = list(object({
type = string
value = number
periodSeconds = number
}))
})
scaleDown = object({
stabilizationWindowSeconds = number
selectPolicy = string
policies = list(object({
type = string
value = number
period_seconds = number
}))
})
})
metrics = list(object({
type = string

resource = optional(object({
name = string
target = object({
type = string
averageUtilization = optional(number)
averageValue = optional(string)
value = optional(string)
})
}))
pods = optional(object({
metric = object({
name = string
selector = optional(object({
matchLabels = optional(map(string))
}))
})
target = object({
type = string
averageValue = optional(string)
value = optional(string)
})
}))
object = optional(object({
metric = object({
name = string
selector = optional(object({
matchLabels = optional(map(string))
}))
})
describedObject = object({
apiVersion = string
kind = string
name = string
})
target = object({
type = string
averageValue = optional(string)
value = optional(string)
})
}))
external = optional(object({
metric = object({
name = string
selector = optional(object({
matchLabels = optional(map(string))
}))
})
target = optional(object({
type = string
averageValue = optional(string)
value = optional(string)
}))
}))
containerResource = optional(object({
name = string
container = string
target = object({
type = string
averageUtilization = optional(number)
averageValue = optional(string)
value = optional(string)
})
}))
}))
})
null no
image The docker gitlab runner image.
object({
registry : optional(string, "registry.gitlab.com")
image : optional(string, "gitlab-org/gitlab-runner")
tag : optional(string)
})
{} no
imagePullPolicy Specify the job images pull policy: Never, IfNotPresent, Always. string "IfNotPresent" no
imagePullSecrets A array of secrets that are used to authenticate Docker image pulling.
list(object({
name = string
}))
null no
livenessProbe n/a
object({
initialDelaySeconds = optional(number, 60)
periodSeconds = optional(number, 10)
successThreshold = optional(number, 1)
failureThreshold = optional(number, 3)
terminationGracePeriodSeconds = optional(number, 30)
})
{} no
logFormat Specifies the log format. Options are runner, text, and json. This setting has lower priority than the format set by command-line argument --log-format. The default value is runner, which contains ANSI escape codes for coloring. string "runner" no
logLevel Configure GitLab Runner's logging level. Available values are: debug, info, warn, error, fatal, panic. string "info" no
metrics Configure integrated Prometheus metrics exporter.
object({
enabled : optional(bool, false)
portName : optional(string, "metrics")
port : optional(number, 9252)
serviceMonitor : optional(object({
enabled : optional(bool, false)
labels : optional(map(string), {})
annotations : optional(map(string), {})
interval : optional(string, "1m")
scheme : optional(string, "http")
tlsConfig : optional(map(string), {})
path : optional(string, "/metrics")
metricRelabeling : optional(list(string), [])
relabelings : optional(list(string), [])
}), {})
})
{} no
nodeSelector A map of node selectors to apply to the pods map(string) {} no
podAnnotations Annotations to be added to the runner pods. map(string) {} no
podLabels Labels to be added to the runner pods. map(string) {} no
podSecurityContext Runner ecurity context for the whole POD.
object({
runAsUser : optional(number, 100)
runAsGroup : optional(number, 65533)
fsGroup : optional(number, 65533)
supplementalGroups : optional(list(number), [65533])
})
{} no
preEntryScript A custom bash script that will be executed prior to the invocation of the gitlab-runner process string null no
priorityClassName Configure priorityClassName for the runner pod. If not set, globalDefault priority class is used. string "" no
rbac RBAC support.
object({
create : optional(bool, false) #create k8s SA and apply RBAC roles #depreciated

rules : optional(list(object({ # Define list of rules to be added to the rbac role permissions.
resources : optional(list(string), []) #resources : optional(list(string), ["pods", "pods/exec", "pods/attach", "secrets", "configmaps"])
apiGroups : optional(list(string), [""])
verbs : optional(list(string)) #verbs : optional(list(string), ["get", "list", "watch", "create", "patch", "delete"])
})), [])

clusterWideAccess : optional(bool, false)

podSecurityPolicy : optional(object({
enabled : optional(bool, false)
resourceNames : optional(list(string), [])
}), {})
})
{} no
readinessProbe n/a
object({
initialDelaySeconds = optional(number, 60)
periodSeconds = optional(number, 10)
successThreshold = optional(number, 1)
failureThreshold = optional(number, 3)
})
{} no
replicas The number of runner pods to create. number 1 no
resources The CPU and memory resources given to the runner.
object({
requests = optional(object({
cpu = optional(string)
memory = optional(string)
ephemeral-storage = optional(string)
})),
limits = optional(object({
cpu = optional(string)
memory = optional(string)
ephemeral-storage = optional(string)
}))
})
null no
runnerToken The Runner Token for adding new Runners to the GitLab Server. string n/a yes
runners n/a
object({
name = string
configPath = optional(string, "") // "Absolute path for an existing runner configuration file"
secret = optional(string, null) // "Secret name containing the runner registration token"
cache = optional(map(string), {}) //Distributed cache secret
config = list(object({
executor = optional(string, "kubernetes")
shell = optional(string, "bash")

environment = optional(list(string), null)
cache_dir = optional(string)

unhealthy_requests_limit = optional(number, 30) //The number of unhealthy responses to new job requests after which a runner worker will be disabled.
unhealthy_interval = optional(string, "120s") //Duration that a runner worker is disabled for after it exceeds the unhealthy requests limit.
output_limit = optional(number, 4096) //Maximum build log size in kilobytes. Default is 4096 (4MB).


kubernetes = object({
namespace = string
pod_labels = optional(map(string), null) // job's pods labels
pod_labels_overwrite_allowed = optional(string, null) //Regular expression to validate the contents of the pod labels overwrite environment variable. When empty, it disables the pod labels overwrite feature.
pod_annotations = optional(map(string), null) // job's annotations

poll_interval : optional(number, 3) //How frequently, in seconds, the runner will poll the Kubernetes pod it has just created to check its status
poll_timeout : optional(number, 180) //The amount of time, in seconds, that needs to pass before the runner will time out attempting to connect to the container it has just created.

image = optional(string) //The image to run jobs with.
helper_image = optional(string) //The default helper image used to clone repositories and upload artifacts.
helper_image_flavor = optional(string) //Sets the helper image flavor (alpine, alpine3.16, alpine3.17, alpine3.18, alpine3.19, alpine-latest, ubi-fips or ubuntu). Defaults to alpine. The alpine flavor uses the same version as alpine3.19.
helper_image_autoset_arch_and_os = optional(string) //Uses the underlying OS to set the Helper Image ARCH and OS.

image_pull_secrets = optional(list(string), null) // An array of items containing the Kubernetes docker-registry secret names used to authenticate Docker image pulling from private registries.
pull_policy = optional(string, "if-not-present") //The Kubernetes pull policy for the runner container. Defaults to if-not-present.
privileged = optional(bool, false) //Whether to run job's podss containers in privileged mode. Defaults to false.

// cpu requests and limits
cpu_limit : optional(string)
cpu_limit_overwrite_max_allowed : optional(string)
cpu_request : optional(string)
cpu_request_overwrite_max_allowed : optional(string)
memory_limit : optional(string)
memory_limit_overwrite_max_allowed : optional(string)
memory_request : optional(string)
memory_request_overwrite_max_allowed : optional(string)
ephemeral_storage_limit : optional(string)
ephemeral_storage_limit_overwrite_max_allowed : optional(string)
ephemeral_storage_request : optional(string)
ephemeral_storage_request_overwrite_max_allowed : optional(string)

//helper containers
helper_cpu_limit : optional(string)
helper_cpu_limit_overwrite_max_allowed : optional(string)
helper_cpu_request : optional(string)
helper_cpu_request_overwrite_max_allowed : optional(string)
helper_memory_limit : optional(string)
helper_memory_limit_overwrite_max_allowed : optional(string)
helper_memory_request : optional(string)
helper_memory_request_overwrite_max_allowed : optional(string)
helper_ephemeral_storage_limit : optional(string)
helper_ephemeral_storage_limit_overwrite_max_allowed : optional(string)
helper_ephemeral_storage_request : optional(string)
helper_ephemeral_storage_request_overwrite_max_allowed : optional(string)

// service containers
service_cpu_limit : optional(string)
service_cpu_limit_overwrite_max_allowed : optional(string)
service_cpu_request : optional(string)
service_cpu_request_overwrite_max_allowed : optional(string)
service_memory_limit : optional(string)
service_memory_limit_overwrite_max_allowed : optional(string)
service_memory_request : optional(string)
service_memory_request_overwrite_max_allowed : optional(string)
service_ephemeral_storage_limit : optional(string)
service_ephemeral_storage_limit_overwrite_max_allowed : optional(string)
service_ephemeral_storage_request : optional(string)
service_ephemeral_storage_request_overwrite_max_allowed : optional(string)

pod_security_context = optional(object({
fs_group : optional(number)
run_as_group : optional(number)
run_as_non_root : optional(bool)
run_as_user : optional(number)
supplemental_groups : optional(list(number))
selinux_options : optional(string)
}), null)

node_selector = optional(map(string), null) //Job PODs node selector
node_tolerations = optional(map(string), null) //Job PODs node tolerations

volumes = optional(object({
empty_dir = optional(list(object({
name = string
mount_path = string
medium = optional(string, null)
size_limit = optional(string, null)
})), null)
host_path = optional(list(object({
name = string
mount_path = string
host_path = string
read_only = optional(bool, false)
})), null)
pvc = optional(list(object({
name = string
mount_path = string
read_only = optional(bool, false)
})), null)
config_map = optional(list(object({
name = string
mount_path = string
read_only = optional(bool, false)
items = optional(map(string), null)
})), null)
secret = optional(list(object({
name = string
mount_path = string
read_only = optional(bool, false)
items = optional(map(string), null)
})), null)
csi = optional(list(object({
name = string
mount_path = string
read_only = optional(bool, false)
driver = string
volume_attributes = optional(map(string), null)
})), null)
}), null)

service_account = optional(string, null) //Default service account job/executor pods use to talk to Kubernetes API. If not set, the default service account is used.
service_account_overwrite_allowed = optional(string, null) //Regular expression to validate the contents of the service account overwrite environment variable. When empty, it disables the service account overwrite feature.

affinity = optional(object({
node_affinity : optional(object({
preferred_during_scheduling_ignored_during_execution : optional(list(object({
weight : number
preference : object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_fields : optional(list(object({
key : string
operator : string
values : list(string)
})))
})
})), null)
required_during_scheduling_ignored_during_execution : optional(list(object({
node_selector_terms : object({
match_expressions : optional(object({
key : string
operator : string
values : list(string)
}))
match_fields : optional(object({
key : string
operator : string
values : list(string)
}))
})
})), null)
}), null)

pod_affinity : optional(object({
preferred_during_scheduling_ignored_during_execution : optional(list(object({
pod_affinity_term : object({
weight : number
topology_key : string
namespaces : optional(list(string))
label_selector : optional(object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_labels : optional(list(string))
}))
namespace_selector : optional(object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_labels : optional(list(string))
}))
})
})), null)
required_during_scheduling_ignored_during_execution : optional(list(object({
topology_key : string
namespaces : optional(list(string))
label_selector : optional(object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_labels : optional(list(string))
}))
namespace_selector : optional(object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_labels : optional(list(string))
}))
})), null)
}), null)

pod_anti_affinity : optional(object({
preferred_during_scheduling_ignored_during_execution : optional(list(object({
pod_affinity_term : object({
weight : number
topology_key : string
namespaces : optional(list(string))
label_selector : optional(object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_labels : optional(list(string))
}))
namespace_selector : optional(object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_labels : optional(list(string))
}))
})
})), null)
required_during_scheduling_ignored_during_execution : optional(list(object({
topology_key : string
namespaces : optional(list(string))
label_selector : optional(object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_labels : optional(list(string))
}))
namespace_selector : optional(object({
match_expressions : optional(list(object({
key : string
operator : string
values : list(string)
})))
match_labels : optional(list(string))
}))
})), null)
}), null)
}), null) //affinity
}) //kubernetes

// start of cache block
cache = optional(object({
Type = optional(string, "gcs")
Path = optional(string, "")
Shared = optional(bool)
MaxUploadedArchiveSize = optional(number)
gcs = optional(object({
CredentialsFile : optional(string)
AccessId : optional(string)
PrivateKey : optional(string)
BucketName : string
}), null)
s3 = optional(map(any), null) //TODO: add static typing as for gcs
azure = optional(map(any), null) //TODO: add static typing as for gcs
secret_name = optional(string)
}), null)
// end of cache block
})) })
n/a yes
schedulerName The name of the scheduler to use. string null no
secrets Secrets to be additionally mounted to the containers.
list(object({
name = string
items = optional(list(object({
key = string
path = string
})))
}))
[] no
securityContext Runner container security context.
object({
allowPrivilegeEscalation : optional(bool, false)
readOnlyRootFilesystem : optional(bool, false)
runAsNonRoot : optional(bool, true)
privileged : optional(bool, false)
capabilities : optional(object({
add : optional(list(string), [])
drop : optional(list(string), [])
}), { drop : ["ALL"] })
})
{} no
sentryDsn Configure GitLab Runner's Sentry DSN. string null no
service Configure a service resource e.g., to allow scraping metrics via prometheus-operator serviceMonitor.
object({
enabled : optional(bool, false)
labels : optional(map(string), {})
annotations : optional(map(string), {})
clusterIP : optional(string, "")
externalIPs : optional(list(string), [])
loadBalancerIP : optional(string, "")
loadBalancerSourceRanges : optional(list(string), [])
type : optional(string, "ClusterIP")
metrics : optional(object({
nodePort : optional(string, "")
}), null),
additionalPorts : optional(list(string), [])
})
{} no
serviceAccount The name of the k8s service account to create (since 17.x.x)
object({
create = optional(bool, false)
name = optional(string, "")
annotations = optional(map(string), {})
imagePullSecrets = optional(list(string), [])
})
{} no
sessionServer Configuration for the session server
object({
enabled = optional(bool, false)
annotations = optional(map(string), null)
timeout = optional(number, null)
internalPort = optional(number, null)
externalPort = optional(number, null)
nodePort = optional(number, null)
publicIP = optional(string, null)
loadBalancerSourceRanges = optional(list(string), null)
serviceType = optional(string, null)
})
{} no
shutdown_timeout Number of seconds until the forceful shutdown operation times out and exits the process. The default value is 30. If set to 0 or lower, the default value is used. number 0 no
strategy Configure update strategy for multi-replica deployments
object({
type = string
rollingUpdate = optional(object({
maxSurge = string
maxUnavailable = string
}), null)
})
null no
terminationGracePeriodSeconds When stopping the runner, give it time (in seconds) to wait for its jobs to terminate. number 3600 no
tolerations List of node taints to tolerate by the runner PODs.
list(object({
key : string
operator : string
value : string
effect : string
}))
[] no
topologySpreadConstraints TopologySpreadConstraints for pod assignment.
list(object({
maxSkew : number
minDomain : optional(number, 1)
topologyKey : string
whenUnsatisfiable : string
labelSelector : object({
matchLabels : optional(map(string), {})
matchExpressions : optional(list(object({
key : string
operator : string
values : list(string)
})), [])
})
matchLabelKeys : optional(list(string), null)
nodeAffinityPolicy : optional(string, null)
nodeTaintsPolicy : optional(string, null)
}))
null no
unregisterRunners Unregister runners before termination. bool true no
useTiny Use the tiny runner image bool false no
values Additional values to be passed to the gitlab-runner helm chart map(any) {} no
values_file Path to Values file to be passed to gitlab-runner helm chart string null no
volumeMounts Additional volumeMounts to add to the runner container.
list(object({
mountPath : string
name : string
mountPropagation : optional(string)
readOnly : optional(bool, false)
subPath : optional(string)
subPathExpr : optional(string)
}))
[] no
volumes List of volumes to be attached to the pod
list(object({
name = string
type = string
options = map(string)
localPath = optional(string)
hostPath = optional(object({
path = string
type = optional(string)
}))
configMap = optional(object({
name = string
items = optional(list(object({
key = string
path = string
})))
}))
secret = optional(object({
secretName = string
items = optional(list(object({
key = string
path = string
})))
}))
emptyDir = optional(object({
medium = optional(string)
sizeLimit = optional(string)
}))
}))
[] no

Outputs

Name Description
helm_release n/a
helm_values n/a
runners n/a

License

This project is free to use and distribute under the MIT License. See LICENSE for more information.

About

This module provides an easy way to deploy GitLab Runners on Kubernetes using Terraform using official Helm chart..

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages