From de31f0141b83faffd202ec97db85e5f08d4015a2 Mon Sep 17 00:00:00 2001 From: Richard Wall Date: Fri, 28 Jun 2024 16:24:06 +0100 Subject: [PATCH] Add metrics settings to the Helm chart Signed-off-by: Richard Wall --- README.md | 10 ++++ .../charts/venafi-kubernetes-agent/README.md | 11 +++++ .../templates/deployment.yaml | 8 ++++ .../templates/podmonitor.yaml | 40 ++++++++++++++++ .../venafi-kubernetes-agent/values.yaml | 46 +++++++++++++++++++ 5 files changed, 115 insertions(+) create mode 100644 deploy/charts/venafi-kubernetes-agent/templates/podmonitor.yaml diff --git a/README.md b/README.md index f87a5d4e..e7dbff74 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,16 @@ go run main.go echo The Jetstack-Secure agent exposes its metrics through a Prometheus server, on port 8081. The Prometheus server is disabled by default but can be enabled by passing the `--enable-metrics` flag to the agent binary. +If you deploy the agent with Helm, using the venafi-kubernetes-agent Helm chart, the metrics server will be enabled by default, on port 8081. +If you use the Prometheus Operator, you can use `--set metrics.podmonitor.enabled=true` to deploy a `PodMonitor` resource, +which will add the venafi-kubernetes-agent metrics to your Prometheus server. + +The following metrics are collected: + * Go collector: via the [default registry](https://github.com/prometheus/client_golang/blob/34e02e282dc4a3cb55ca6441b489ec182e654d59/prometheus/registry.go#L60-L63) in Prometheus client_golang. + * Process collector: via the [default registry](https://github.com/prometheus/client_golang/blob/34e02e282dc4a3cb55ca6441b489ec182e654d59/prometheus/registry.go#L60-L63) in Prometheus client_golang. + * Agent metrics: + * `data_readings_upload_size`: Data readings upload size (in bytes) sent by the jscp in-cluster agent. + ## Release Process The release process is semi-automated. diff --git a/deploy/charts/venafi-kubernetes-agent/README.md b/deploy/charts/venafi-kubernetes-agent/README.md index a5ef0ebf..3b50cdb8 100644 --- a/deploy/charts/venafi-kubernetes-agent/README.md +++ b/deploy/charts/venafi-kubernetes-agent/README.md @@ -159,6 +159,15 @@ You should see the following events for your service account: | image.repository | string | `"registry.venafi.cloud/venafi-agent/venafi-agent"` | Default to Open Source image repository | | image.tag | string | `"v0.1.48"` | Overrides the image tag whose default is the chart appVersion | | imagePullSecrets | list | `[]` | Specify image pull credentials if using a private registry example: - name: my-pull-secret | +| metrics.enabled | bool | `true` | Enable the metrics server. If false, the metrics server will be disabled and the other metrics fields below will be ignored. | +| metrics.podmonitor.annotations | object | `{}` | Additional annotations to add to the PodMonitor. | +| metrics.podmonitor.enabled | bool | `false` | Create a PodMonitor to add the metrics to Prometheus, if you are using Prometheus Operator. See https://prometheus-operator.dev/docs/operator/api/#monitoring.coreos.com/v1.PodMonitor | +| metrics.podmonitor.endpointAdditionalProperties | object | `{}` | EndpointAdditionalProperties allows setting additional properties on the endpoint such as relabelings, metricRelabelings etc. For example: endpointAdditionalProperties: relabelings: - action: replace sourceLabels: - __meta_kubernetes_pod_node_name targetLabel: instance | +| metrics.podmonitor.honorLabels | bool | `false` | Keep labels from scraped data, overriding server-side labels. | +| metrics.podmonitor.interval | string | `"60s"` | The interval to scrape metrics. | +| metrics.podmonitor.labels | object | `{}` | Additional labels to add to the PodMonitor. | +| metrics.podmonitor.prometheusInstance | string | `"default"` | Specifies the `prometheus` label on the created PodMonitor. This is used when different Prometheus instances have label selectors matching different PodMonitors. | +| metrics.podmonitor.scrapeTimeout | string | `"30s"` | The timeout before a metrics scrape fails. | | nameOverride | string | `""` | Helm default setting to override release name, usually leave blank. | | nodeSelector | object | `{}` | Embed YAML for nodeSelector settings, see https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/ | | podAnnotations | object | `{}` | Additional YAML annotations to add the the pod. | @@ -172,4 +181,6 @@ You should see the following events for your service account: | serviceAccount.create | bool | `true` | Specifies whether a service account should be created | | serviceAccount.name | string | `""` | The name of the service account to use. If blank and `serviceAccount.create` is true, a name is generated using the fullname template of the release. | | tolerations | list | `[]` | Embed YAML for toleration settings, see https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ | +| volumeMounts | list | `[]` | Additional volume mounts to add to the Venafi Kubernetes Agent container. This is useful for mounting a custom CA bundle. Any PEM certificate mounted under /etc/ssl/certs will be loaded by the Venafi Kubernetes Agent. For example: volumeMounts: - name: cabundle mountPath: /etc/ssl/certs/cabundle subPath: cabundle readOnly: true | +| volumes | list | `[]` | Additional volumes to add to the Venafi Kubernetes Agent container. This is useful for mounting a custom CA bundle. For example: volumes: - name: cabundle configMap: name: cabundle optional: false defaultMode: 0644 In order to create the ConfigMap, you can use the following command: kubectl create configmap cabundle \ --from-file=cabundle=./your/custom/ca/bundle.pem | diff --git a/deploy/charts/venafi-kubernetes-agent/templates/deployment.yaml b/deploy/charts/venafi-kubernetes-agent/templates/deployment.yaml index d9894285..68a1b9bb 100644 --- a/deploy/charts/venafi-kubernetes-agent/templates/deployment.yaml +++ b/deploy/charts/venafi-kubernetes-agent/templates/deployment.yaml @@ -62,6 +62,9 @@ spec: - "-p" - "0h1m0s" - --venafi-cloud + {{- if .Values.metrics.enabled }} + - --enable-metrics + {{- end }} {{- range .Values.extraArgs }} - {{ . | quote }} {{- end }} @@ -77,6 +80,11 @@ spec: {{- with .Values.volumeMounts }} {{- toYaml . | nindent 12 }} {{- end }} + {{- if .Values.metrics.enabled }} + ports: + - containerPort: 8081 + name: http-metrics + {{- end }} {{- with .Values.nodeSelector }} nodeSelector: {{- toYaml . | nindent 8 }} diff --git a/deploy/charts/venafi-kubernetes-agent/templates/podmonitor.yaml b/deploy/charts/venafi-kubernetes-agent/templates/podmonitor.yaml new file mode 100644 index 00000000..7d798d75 --- /dev/null +++ b/deploy/charts/venafi-kubernetes-agent/templates/podmonitor.yaml @@ -0,0 +1,40 @@ +{{- if and .Values.metrics.enabled .Values.metrics.podmonitor.enabled }} +apiVersion: monitoring.coreos.com/v1 +kind: PodMonitor +metadata: + name: {{ include "venafi-kubernetes-agent.fullname" . }} +{{- if .Values.metrics.podmonitor.namespace }} + namespace: {{ .Values.metrics.podmonitor.namespace }} +{{- else }} + namespace: {{ .Release.Namespace | quote }} +{{- end }} + labels: + {{- include "venafi-kubernetes-agent.labels" . | nindent 4 }} + prometheus: {{ .Values.metrics.podmonitor.prometheusInstance }} + {{- with .Values.metrics.podmonitor.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +{{- with .Values.metrics.podmonitor.annotations }} + annotations: + {{- toYaml . | nindent 4 }} +{{- end }} +spec: + jobLabel: {{ include "venafi-kubernetes-agent.fullname" . }} + selector: + matchLabels: + {{- include "venafi-kubernetes-agent.selectorLabels" . | nindent 6 }} +{{- if .Values.metrics.podmonitor.namespace }} + namespaceSelector: + matchNames: + - {{ .Release.Namespace | quote }} +{{- end }} + podMetricsEndpoints: + - port: http-metrics + path: /metrics + interval: {{ .Values.metrics.podmonitor.interval }} + scrapeTimeout: {{ .Values.metrics.podmonitor.scrapeTimeout }} + honorLabels: {{ .Values.metrics.podmonitor.honorLabels }} + {{- with .Values.metrics.podmonitor.endpointAdditionalProperties }} + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/deploy/charts/venafi-kubernetes-agent/values.yaml b/deploy/charts/venafi-kubernetes-agent/values.yaml index f2780fd0..6acbbaef 100644 --- a/deploy/charts/venafi-kubernetes-agent/values.yaml +++ b/deploy/charts/venafi-kubernetes-agent/values.yaml @@ -2,6 +2,52 @@ # This is a YAML-formatted file. # Declare variables to be passed into your templates. +metrics: + # -- Enable the metrics server. + # If false, the metrics server will be disabled and the other metrics fields below will be ignored. + enabled: true + podmonitor: + # -- Create a PodMonitor to add the metrics to Prometheus, if you are using Prometheus Operator. + # See https://prometheus-operator.dev/docs/operator/api/#monitoring.coreos.com/v1.PodMonitor + enabled: false + + # -- The namespace that the pod monitor should live in. + # Defaults to the venafi-kubernetes-agent namespace. + # +docs:property + # namespace: venafi + + # -- Specifies the `prometheus` label on the created PodMonitor. + # This is used when different Prometheus instances have label selectors + # matching different PodMonitors. + prometheusInstance: default + + # -- The interval to scrape metrics. + interval: 60s + + # -- The timeout before a metrics scrape fails. + scrapeTimeout: 30s + + # -- Additional labels to add to the PodMonitor. + labels: {} + + # -- Additional annotations to add to the PodMonitor. + annotations: {} + + # -- Keep labels from scraped data, overriding server-side labels. + honorLabels: false + + # -- EndpointAdditionalProperties allows setting additional properties on the endpoint such as relabelings, metricRelabelings etc. + # + # For example: + # endpointAdditionalProperties: + # relabelings: + # - action: replace + # sourceLabels: + # - __meta_kubernetes_pod_node_name + # targetLabel: instance + # + endpointAdditionalProperties: {} + # -- default replicas, do not scale up replicaCount: 1