Jenkins Configuration as Code (helm#9057)

* Includes Jenkins Configuration as Code Signed-off-by: Brendan Holmes <[email protected]> * Minor README.md changes Signed-off-by: Brendan Holmes <[email protected]> * bumped CasC plugin version Signed-off-by: Brendan Holmes <[email protected]> * Adding auto config reload Signed-off-by: Brendan Holmes <[email protected]> * Needed to exclude some deployment config in the without-jcasc case. Fixed tabulation and simplified jcasc example in values.yaml. Tweaked documentation. Signed-off-by: Brendan Holmes <[email protected]> * Disabling auto-reload by default. Signed-off-by: Brendan Holmes <[email protected]> * Fixes error in corner-case where user has disabled config as code, but enabled auto-config. Signed-off-by: Brendan Holmes <[email protected]> * Expanded auto-config info in readme and added guidance for using non-internal identity db. Doubled master memory limit since entered OOM restart loop when using config-as-code plugin. Fixed missing casc_configs dir (in config.yaml & deployment.yaml) when not using auto-reload Signed-off-by: Brendan Holmes <[email protected]> * Sidecar was reloading once per key in the configmap when any single key had changed. Resolved by creating separate configmaps, one for each key under ConfigScripts. I was mistaken above that users only need Overall\Read rights to auto-reload. Seems JCasC has higher privilege requirements than the CLI\API generally. I've amended the Readme accordingly. Process for enabling for LDAP\other ID store is still simple. Fixed connectivity from sidecar when enabling non-root privileges by using a TCP port > 1024 (1044) Bumped the sidecar image from 0.0.1 to 0.0.2 which a few improvements: faster, less error-prone startup by testing the Jenkins container's avaibility using SSH port instead of the main jenkins port. This removes the need for an arbitary wait. Also fixed "access denied" when enabling non-root privileges by creating the same jenkins 1000 user in the sidecar. Signed-off-by: Brendan Holmes <[email protected]> * Minor fix: configmap names now include the release name Signed-off-by: Brendan Holmes <[email protected]> * Update stable/jenkins/templates/config.yaml Co-Authored-By: holmesb <[email protected]> Signed-off-by: Brendan Holmes <[email protected]> * Update stable/jenkins/templates/jcasc_config.yaml Co-Authored-By: holmesb <[email protected]> Signed-off-by: Brendan Holmes <[email protected]> * Replaced a few if conditions with simpler else statements in config.yaml Signed-off-by: Brendan Holmes <[email protected]> * Bumping to the latest Config as Code plugin version. Signed-off-by: Brendan Holmes <[email protected]>
wmcdona89 · Feb 8, 2019 · 4319bab · 4319bab
1 parent 0d1d000
commit 4319bab
Show file tree

Hide file tree

Showing 9 changed files with 316 additions and 46 deletions.
diff --git a/charts/jenkins/Chart.yaml b/charts/jenkins/Chart.yaml
@@ -1,6 +1,6 @@
 name: jenkins
 home: https://jenkins.io/
-version: 0.28.11
+version: 0.29.0
 appVersion: lts
 description: Open source continuous integration server. It supports multiple SCM tools
   including CVS, Subversion and Git. It can execute Apache Ant and Apache Maven-based
@@ -9,6 +9,7 @@ sources:
 - https://github.com/jenkinsci/jenkins
 - https://github.com/jenkinsci/docker-jnlp-slave
 - https://github.com/nuvo/kube-tasks
+- https://github.com/jenkinsci/configuration-as-code-plugin
 maintainers:
 - name: lachie83
   email: [email protected]

diff --git a/charts/jenkins/README.md b/charts/jenkins/README.md
@@ -69,12 +69,12 @@ The following tables list the configurable parameters of the Jenkins chart and t
 | `Master.LoadBalancerIP`           | Optional fixed external IP           | Not set                                                                      |
 | `Master.JMXPort`                  | Open a port, for JMX stats           | Not set                                                                      |
 | `Master.ExtraPorts`               | Open extra ports, for other uses     | Not set                                                                      |
-| `Master.CustomConfigMap`          | Use a custom ConfigMap               | `false`                                                                      |
-| `Master.AdditionalConfig`          | Add additional config files         | `{}`                                                                      |
 | `Master.OverwriteConfig`          | Replace config w/ ConfigMap on boot  | `false`                                                                      |
 | `Master.Ingress.Annotations`      | Ingress annotations                  | `{}`                                                                         |
 | `Master.Ingress.Path`             | Ingress path                         | Not set                                                                         |
 | `Master.Ingress.TLS`              | Ingress TLS configuration            | `[]`                                                                         |
+| `Master.JCasC.ConfigScripts`      | List of Jenkins Config as Code scripts | False                                                                   |
+| `Master.Sidecar.configAutoReload` | Jenkins Config as Code auto-reload settings | False                                                                    |
 | `Master.InitScripts`              | List of Jenkins init scripts         | Not set                                                                      |
 | `Master.CredentialsXmlSecret`     | Kubernetes secret that contains a 'credentials.xml' file | Not set                                                  |
 | `Master.SecretsFilesSecret`       | Kubernetes secret that contains 'secrets' files | Not set                                                           |
@@ -86,6 +86,8 @@ The following tables list the configurable parameters of the Jenkins chart and t
 | `Master.Affinity`                 | Affinity settings                    | `{}`                                                                         |
 | `Master.Tolerations`              | Toleration labels for pod assignment | `{}`                                                                         |
 | `Master.PodAnnotations`           | Annotations for master pod           | `{}`                                                                         |
+| `Master.CustomConfigMap`          | Deprecated: Use a custom ConfigMap               | `false`                                                                      |
+| `Master.AdditionalConfig`         | Deprecated: Add additional config files         | `{}`                                                                      |
 | `NetworkPolicy.Enabled`           | Enable creation of NetworkPolicy resources. | `false`                                                               |
 | `NetworkPolicy.ApiVersion`        | NetworkPolicy ApiVersion             | `networking.k8s.io/v1`                                                         |
 | `rbac.install`                    | Create service account and ClusterRoleBinding for Kubernetes plugin | `false`                                       |
@@ -215,25 +217,41 @@ It is possible to mount several volumes using `Persistence.volumes` and `Persist
 $ helm install --name my-release --set Persistence.ExistingClaim=PVC_NAME stable/jenkins
 ```
 
-## Custom ConfigMap
-
-When creating a new parent chart with this chart as a dependency, the `CustomConfigMap` parameter can be used to override the default config.xml provided.
-It also allows for providing additional xml configuration files that will be copied into `/var/jenkins_home`. In the parent chart's values.yaml,
-set the `jenkins.Master.CustomConfigMap` value to true like so
+## Configuration as Code
+Jenkins Configuration as Code is now a standard component in the Jenkins project.  Add a key under ConfigScripts for each configuration area, where each corresponds to a plugin or section of the UI.  The keys (prior to | character) are just labels, and can be any value.  They are only used to give the section a meaningful name.  The only restriction is they must conform to RFC 1123 definition of a DNS label, so may only contain lowercase letters, numbers, and hyphens.  Each key will become the name of a configuration yaml file on the master in /var/jenkins_home/casc_configs (by default) and will be processed by the Configuration as Code Plugin during Jenkins startup.  The lines after each | become the content of the configuration yaml file.  The first line after this is a JCasC root element, eg jenkins, credentials, etc.  Best reference is the Documentation link here: https://<jenkins_url>/configuration-as-code.  The example below creates ldap settings:
 
 ```yaml
-jenkins:
-  Master:
-    CustomConfigMap: true
+ConfigScripts:
+  ldap-settings: |
+    jenkins:
+      securityRealm:
+        ldap:
+          configurations:
+            configurations:
+              - server: ldap.acme.com
+                rootDN: dc=acme,dc=uk
+                managerPasswordSecret: ${LDAP_PASSWORD}
+              - groupMembershipStrategy:
+                  fromUserRecord:
+                    attributeName: "memberOf"
 ```
 
-and provide the file `templates/config.tpl` in your parent chart for your use case. You can start by copying the contents of `config.yaml` from this chart into your parent charts `templates/config.tpl` as a basis for customization. Finally, you'll need to wrap the contents of `templates/config.tpl` like so:
+Further JCasC examples can be found [here.](https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos)
+### Config as Code with and without auto-reload 
+Config as Code changes (to Master.JCasC.ConfigScripts) can either force a new pod to be created and only be applied at next startup, or can be auto-reloaded on-the-fly.  If you choose `Master.Sidecar.autoConfigReload.enabled: true`, a second, auxiliary container will be installed into the Jenkins master pod, known as a "sidecar".  This watches for changes to ConfigScripts, copies the content onto the Jenkins file-system and issues a CLI command via SSH to reload configuration.  The admin user (or account you specify in Master.AdminUser) will have a random SSH private key (RSA 4096) assigned unless you specify `Master.OwnSshKey: true`.  This will be saved to a k8s secret.  You can monitor this sidecar's logs using command `kubectl logs <master_pod> -c jenkins-sc-config -f`
 
+### Auto-reload with non-Jenkins identities
+When enabling LDAP or another non-Jenkins identity source, the built-in admin account will no longer exist.  Since the admin account is used by the sidecar to reload config, in order to use auto-reload, you must change the .Master.AdminUser to a valid username on your LDAP (or other) server.  If you use the matrix-auth plugin, this user must also be granted Overall\Administer rights in Jenkins.  Failure to do this will cause the sidecar container to fail to authenticate via SSH and enter a restart loop.  You can enable LDAP using the example above and add a Config as Code block for matrix security that includes:
 ```yaml
-{{- define "override_config_map" }}
-    <CONTENTS_HERE>
-{{ end }}
+ConfigScripts:
+	matrix-auth: |
+		Jenkins:
+          authorizationStrategy:
+            projectMatrix:
+              grantedPermissions:
+			    - "Overall/Administer:<AdminUser_LDAP_username>"
 ```
+You can instead grant this permission via the UI.  When this is done, you can set `Master.Sidecar.configAutoReload.enabled: true` and upon the next Helm upgrade, auto-reload will be successfully enabled.
 
 ## RBAC
 
@@ -366,3 +384,25 @@ Master:
     -Dhttps.proxyHost=192.168.64.1
     -Dhttps.proxyPort=3128
 ```
+
+## Custom ConfigMap
+
+The following configuration method is deprecated and will be removed in an upcoming version of this chart.
+We recommend you use Jenkins Configuration as Code to configure instead.
+When creating a new parent chart with this chart as a dependency, the `CustomConfigMap` parameter can be used to override the default config.xml provided.
+It also allows for providing additional xml configuration files that will be copied into `/var/jenkins_home`. In the parent chart's values.yaml,
+set the `jenkins.Master.CustomConfigMap` value to true like so
+
+```yaml
+jenkins:
+  Master:
+    CustomConfigMap: true
+```
+
+and provide the file `templates/config.tpl` in your parent chart for your use case. You can start by copying the contents of `config.yaml` from this chart into your parent charts `templates/config.tpl` as a basis for customization. Finally, you'll need to wrap the contents of `templates/config.tpl` like so:
+
+```yaml
+{{- define "override_config_map" }}
+    <CONTENTS_HERE>
+{{ end }}
+```
diff --git a/charts/jenkins/templates/NOTES.txt b/charts/jenkins/templates/NOTES.txt
@@ -26,9 +26,16 @@
 {{- end }}
 
 3. Login with the password from step 1 and the username: {{ .Values.Master.AdminUser }}
+{{ if .Values.Master.JCasC.enabled }}
+4. Use Jenkins Configuration as Code by specifying ConfigScripts in your values.yaml file, see documentation: http://{{ .Values.Master.HostName }}/configuration-as-code and examples: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos
+{{- end }}
 
 For more information on running Jenkins on Kubernetes, visit:
 https://cloud.google.com/solutions/jenkins-on-container-engine
+{{- if .Values.Master.JCasC.enabled }}
+For more information about Jenkins Configuration as Code, visit:
+https://jenkins.io/projects/jcasc/
+{{- end }}
 
 {{- if .Values.Persistence.Enabled }}
 {{- else }}

diff --git a/charts/jenkins/templates/_helpers.tpl b/charts/jenkins/templates/_helpers.tpl
@@ -32,3 +32,13 @@ If release name contains chart name it will be used as a full name.
     {{- end -}}
   {{- end -}}
 {{- end -}}
+
+{{/*
+Generate private key for jenkins CLI
+*/}}
+{{- define "jenkins.gen-key" -}}
+{{- if not .Values.Master.OwnSshKey -}}
+{{- $key := genPrivateKey "rsa" -}}
+jenkins-admin-private-key: {{ $key | b64enc }}
+{{- end -}}
+{{- end -}}
diff --git a/charts/jenkins/templates/config.yaml b/charts/jenkins/templates/config.yaml
@@ -187,15 +187,15 @@ data:
 {{- if .Values.Master.JenkinsUrl }}
       <jenkinsUrl>{{ .Values.Master.JenkinsUrl }}</jenkinsUrl>
 {{- else }}
-{{- if .Values.Master.HostName }}
-{{- if .Values.Master.Ingress.TLS }}
+  {{- if .Values.Master.HostName }}
+    {{- if .Values.Master.Ingress.TLS }}
       <jenkinsUrl>https://{{ .Values.Master.HostName }}{{ default "" .Values.Master.JenkinsUriPrefix }}</jenkinsUrl>
-{{- else }}
+    {{- else }}
       <jenkinsUrl>http://{{ .Values.Master.HostName }}{{ default "" .Values.Master.JenkinsUriPrefix }}</jenkinsUrl>
-{{- end }}
-{{- else }}
+    {{- end }}
+  {{- else }}
       <jenkinsUrl>http://{{ template "jenkins.fullname" . }}:{{.Values.Master.ServicePort}}{{ default "" .Values.Master.JenkinsUriPrefix }}</jenkinsUrl>
-{{- end}}
+  {{- end}}
 {{- end}}
     </jenkins.model.JenkinsLocationConfiguration>
   jenkins.CLI.xml: |-
@@ -214,15 +214,15 @@ data:
     cp /var/jenkins_config/config.xml /var/jenkins_home;
     cp /var/jenkins_config/jenkins.CLI.xml /var/jenkins_home;
     cp /var/jenkins_config/jenkins.model.JenkinsLocationConfiguration.xml /var/jenkins_home;
-{{- else }}
+  {{- else }}
     yes n | cp -i /var/jenkins_config/config.xml /var/jenkins_home;
     yes n | cp -i /var/jenkins_config/jenkins.CLI.xml /var/jenkins_home;
     yes n | cp -i /var/jenkins_config/jenkins.model.JenkinsLocationConfiguration.xml /var/jenkins_home;
-{{- if .Values.Master.AdditionalConfig }}
+  {{- if .Values.Master.AdditionalConfig }}
 {{- range $key, $val := .Values.Master.AdditionalConfig }}
     cp /var/jenkins_config/{{- $key }} /var/jenkins_home;
-{{- end }}
-{{- end }}
+  {{- end }}
+  {{- end }}
 {{- end }}
 {{- if .Values.Master.InstallPlugins }}
     # Install missing plugins
@@ -235,10 +235,25 @@ data:
 {{- if .Values.Master.ScriptApproval }}
     yes n | cp -i /var/jenkins_config/scriptapproval.xml /var/jenkins_home/scriptApproval.xml;
 {{- end }}
+{{- if and (.Values.Master.JCasC.enabled) (.Values.Master.Sidecar.configAutoReload.enabled) }}
+  {{- if not .Values.Master.InitScripts }}
+    mkdir -p /var/jenkins_home/init.groovy.d/;
+    yes n | cp -i /var/jenkins_config/*.groovy /var/jenkins_home/init.groovy.d/;
+  {{- end }}
+{{- end }}
 {{- if .Values.Master.InitScripts }}
     mkdir -p /var/jenkins_home/init.groovy.d/;
     yes n | cp -i /var/jenkins_config/*.groovy /var/jenkins_home/init.groovy.d/;
 {{- end }}
+{{- if .Values.Master.JCasC.enabled }}
+  {{- if .Values.Master.Sidecar.configAutoReload.enabled }}
+    bash -c 'ssh-keygen -y -f <(echo "${ADMIN_PRIVATE_KEY}") > /var/jenkins_home/key.pub'
+  {{- else }}
+    mkdir -p /var/jenkins_home/casc_configs;
+    rm -rf /var/jenkins_home/casc_configs/*
+    cp -v /var/jenkins_config/*.yaml /var/jenkins_home/casc_configs
+  {{- end }}
+{{- end }}
 {{- if .Values.Master.CredentialsXmlSecret }}
     yes n | cp -i /var/jenkins_credentials/credentials.xml /var/jenkins_home;
 {{- end }}
@@ -254,12 +269,45 @@ data:
 {{- range $key, $val := .Values.Master.InitScripts }}
   init{{ $key }}.groovy: |-
 {{ $val | indent 4 }}
+{{- end }}
+{{- if .Values.Master.JCasC.enabled }}
+  {{- if .Values.Master.Sidecar.configAutoReload.enabled }}
+  init-add-ssh-key-to-admin.groovy: |-
+    import jenkins.security.*
+    import hudson.model.User
+    import jenkins.security.ApiTokenProperty
+    import jenkins.model.Jenkins
+    User u = User.get("{{ .Values.Master.AdminUser | default "admin" }}")
+    ApiTokenProperty t = u.getProperty(ApiTokenProperty.class)
+    String sshKeyString = new File('/var/jenkins_home/key.pub').text
+    keys_param = new org.jenkinsci.main.modules.cli.auth.ssh.UserPropertyImpl(sshKeyString)
+    u.addProperty(keys_param)
+    def inst = Jenkins.getInstance()
+    def sshDesc = inst.getDescriptor("org.jenkinsci.main.modules.sshd.SSHD")
+    sshDesc.setPort({{ .Values.Master.Sidecar.configAutoReload.sshTcpPort | default 1044 }})
+    sshDesc.getActualPort()
+    sshDesc.save()
+  {{- else }}
+# Only add config to this script if we aren't auto-reloading otherwise the pod will restart upon each config change:
+{{- range $key, $val := .Values.Master.JCasC.ConfigScripts }}
+  {{ $key }}.yaml: |-
+{{ tpl $val $| indent 4 }}
+{{- end }}
+{{- end }}
 {{- end }}
   plugins.txt: |-
 {{- if .Values.Master.InstallPlugins }}
 {{- range $index, $val := .Values.Master.InstallPlugins }}
 {{ $val | indent 4 }}
 {{- end }}
+{{- if .Values.Master.JCasC.enabled }}
+  {{- if not (contains "configuration-as-code" (quote .Values.Master.InstallPlugins)) }}
+    configuration-as-code:{{ .Values.Master.JCasC.PluginVersion }}
+  {{- end }}
+  {{- if not (contains "configuration-as-code-support" (quote .Values.Master.InstallPlugins)) }}
+    configuration-as-code-support:{{ .Values.Master.JCasC.SupportPluginVersion }}
+  {{- end }}
+{{- end }}
 {{- end }}
 {{ else }}
 {{ include "override_config_map" . }}

diff --git a/charts/jenkins/templates/jcasc_config.yaml b/charts/jenkins/templates/jcasc_config.yaml
@@ -0,0 +1,18 @@
+{{- $root := . }}
+{{- if and (.Values.Master.JCasC.enabled) (.Values.Master.Sidecar.configAutoReload.enabled) }}
+{{- range $key, $val := .Values.Master.JCasC.ConfigScripts }}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: jenkins-config-{{ template "jenkins.fullname" $root }}-{{ $key }}
+  labels:
+    {{ $.Values.Master.Sidecar.configAutoReload.label | default "jenkins_config" }}: "true"
+    release: {{ $root.Release.Name }}
+    chart: "{{ $root.Chart.Name }}-{{ $root.Chart.Version }}"
+    component: "{{ $root.Release.Name }}-{{ $.Values.Master.Name }}"
+data:
+  {{ $key }}.yaml: |-
+{{ tpl $val $| indent 4 }}
+{{- end }}
+{{- end }}