Merge branch 'main' into add-mimir-continoustest-alerts

giantswarm · Nov 12, 2024 · 13df703 · 13df703
2 parents d6b4ace + de44204
commit 13df703
Show file tree

Hide file tree

Showing 81 changed files with 2,276 additions and 859 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Removed
+
+- Remove the `mimir.enabled` property to replace it with the MC flavor as all CAPI MCs now run Mimir.
+
+## [4.24.1] - 2024-11-12
+
+### Fixed
+
+- Fix `MonitoringAgentDown` to page when both prometheus-agent and alloy-metrics jobs are missing.
+
+## [4.24.0] - 2024-11-12
+
+### Added
+
+- Add a set of sensible alerts to monitor alloy.
+  - `AlloySlowComponentEvaluations` and `AlloyUnhealthyComponents` to report about alloy component state.
+  - `LoggingAgentDown` to be alerted when the logging agent is down.
+  - `LogForwardingErrors` to be alerted when the `loki.write` component is failing.
+  - `LogReceivingErrors` to be alerted when the `loki.source.api` components of the gateway is failing.
+  - `MonitoringAgentDown` to be alerted when the monitoring agent is down.
+  - `MonitoringAgentShardsNotSatisfied` to be alerted when the monitoring agent is missing any number of desired shards.
+
+### Changed
+
+- Update `DeploymentNotSatisfiedAtlas` to take into account the following components:
+  - `observability-operator`
+  - `alloy-rules`
+  - `observability-gateway`
+- Move all `grafana-cloud` related alerts to their own file.
+- Move all alloy related alerts to the alloy alert file.
+- Rename and move the following alerts as they are not specific to Prometheus:
+  - `PrometheusCriticalJobScrapingFailure` => `CriticalJobScrapingFailure`
+  - `PrometheusJobScrapingFailure` => `JobScrapingFailure`
+  - `PrometheusFailsToCommunicateWithRemoteStorageAPI` => `MetricForwardingErrors`
+
 ## [4.23.0] - 2024-10-30
 
 ### Changed
@@ -19,6 +54,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Fixed
 
 - Fixes the statefulset.rules name as it is currently replacing the deployment.rules alerts.
+- Extends AppCR-related alerts with cancelation for CAPI clusters with unavailable control plane.
 
 ## [4.22.0] - 2024-10-29
 
@@ -3190,7 +3226,9 @@ Fix `PromtailRequestsErrors` alerts as promtail retries after some backoff so ac
 
 - Add existing rules from https://github.com/giantswarm/prometheus-meta-operator/pull/637/commits/bc6a26759eb955de92b41ed5eb33fa37980660f2
 
-[Unreleased]: https://github.com/giantswarm/prometheus-rules/compare/v4.23.0...HEAD
+[Unreleased]: https://github.com/giantswarm/prometheus-rules/compare/v4.24.1...HEAD
+[4.24.1]: https://github.com/giantswarm/prometheus-rules/compare/v4.24.0...v4.24.1
+[4.24.0]: https://github.com/giantswarm/prometheus-rules/compare/v4.23.0...v4.24.0
 [4.23.0]: https://github.com/giantswarm/prometheus-rules/compare/v4.22.0...v4.23.0
 [4.22.0]: https://github.com/giantswarm/prometheus-rules/compare/v4.21.1...v4.22.0
 [4.21.1]: https://github.com/giantswarm/prometheus-rules/compare/v4.21.0...v4.21.1

diff --git a/README.md b/README.md
@@ -168,11 +168,11 @@ There are 2 kinds of tests on rules:
    ```
    [...]
    ###  Testing platform/atlas/alerting-rules/prometheus-operator.rules.yml
-   ###    promtool check rules /home/marie/github-repo/prometheus-rules/test/hack/output/generated/capi/capa-mimir/platform/atlas/alerting-rules/prometheus-operator.rules.yml
+   ###    promtool check rules /home/marie/github-repo/prometheus-rules/test/hack/output/generated/capi/capa/platform/atlas/alerting-rules/prometheus-operator.rules.yml
    ###    Skipping platform/atlas/alerting-rules/prometheus-operator.rules.yml: listed in test/conf/promtool_ignore
    ###  Testing platform/atlas/alerting-rules/prometheus.rules.yml
-   ###    promtool check rules /home/marie/github-repo/prometheus-rules/test/hack/output/generated/capi/capa-mimir/platform/atlas/alerting-rules/prometheus.rules.yml
-   ###    promtool test rules prometheus.rules.test.yml - capi/capa-mimir
+   ###    promtool check rules /home/marie/github-repo/prometheus-rules/test/hack/output/generated/capi/capa/platform/atlas/alerting-rules/prometheus.rules.yml
+   ###    promtool test rules prometheus.rules.test.yml - capi/capa
    [...]
    09:06:29 promtool: end (Elapsed time: 1s)
    Congratulations!  Prometheus rules have been promtool checked and tested

diff --git a/helm/prometheus-rules/Chart.yaml b/helm/prometheus-rules/Chart.yaml
@@ -5,7 +5,7 @@ home: https://github.com/giantswarm/prometheus-rules
 icon: https://s.giantswarm.io/app-icons/1/png/default-app-light.png
 name: prometheus-rules
 appVersion: '0.1.0'
-version: '4.23.0'
+version: '4.24.1'
 annotations:
   application.giantswarm.io/team: "atlas"
   config.giantswarm.io/version: 1.x.x
diff --git a/helm/prometheus-rules/templates/alloy-rules-configmap.yaml b/helm/prometheus-rules/templates/alloy-rules-configmap.yaml
@@ -1,4 +1,4 @@
-{{- if .Values.mimir.enabled }}
+{{- if eq .Values.managementCluster.provider.flavor "capi" }}
 apiVersion: v1
 kind: ConfigMap
 metadata:

diff --git a/helm/prometheus-rules/templates/alloy-rules.yaml b/helm/prometheus-rules/templates/alloy-rules.yaml
@@ -1,4 +1,4 @@
-{{- if .Values.mimir.enabled }}
+{{- if eq .Values.managementCluster.provider.flavor "capi" }}
 apiVersion: application.giantswarm.io/v1alpha1
 kind: App
 metadata:

diff --git a/...etheus-rules/templates/kaas/phoenix/alerting-rules/aws-load-balancer-controller.rules.yml b/...etheus-rules/templates/kaas/phoenix/alerting-rules/aws-load-balancer-controller.rules.yml
@@ -5,9 +5,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
-{{- end }}
+    {{- end }}
   name: aws-load-balancer-controller.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/...rometheus-rules/templates/kaas/phoenix/alerting-rules/aws.node.workload-cluster.rules.yml b/...rometheus-rules/templates/kaas/phoenix/alerting-rules/aws.node.workload-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
-{{- end }}
+    {{- end }}
   name: node.aws.workload-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/phoenix/alerting-rules/aws.workload-cluster.rules.yml b/helm/prometheus-rules/templates/kaas/phoenix/alerting-rules/aws.workload-cluster.rules.yml
@@ -5,7 +5,7 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-    {{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
     {{- end }}
   name: aws.workload-cluster.rules

diff --git a/.../prometheus-rules/templates/kaas/phoenix/alerting-rules/capa.management-cluster.rules.yml b/.../prometheus-rules/templates/kaas/phoenix/alerting-rules/capa.management-cluster.rules.yml
@@ -6,9 +6,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: capa.management-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/phoenix/alerting-rules/irsa.rules.yml b/helm/prometheus-rules/templates/kaas/phoenix/alerting-rules/irsa.rules.yml
@@ -3,9 +3,9 @@ kind: PrometheusRule
 metadata:
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: irsa.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/...etheus-rules/templates/kaas/turtles/alerting-rules/apiserver.management-cluster.rules.yml b/...etheus-rules/templates/kaas/turtles/alerting-rules/apiserver.management-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: apiserver.management-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/...ometheus-rules/templates/kaas/turtles/alerting-rules/apiserver.workload-cluster.rules.yml b/...ometheus-rules/templates/kaas/turtles/alerting-rules/apiserver.workload-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
-{{- end }}
+    {{- end }}
   name: apiserver.workload-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/.../prometheus-rules/templates/kaas/turtles/alerting-rules/capi.management-cluster.rules.yml b/.../prometheus-rules/templates/kaas/turtles/alerting-rules/capi.management-cluster.rules.yml
@@ -4,9 +4,9 @@ kind: PrometheusRule
 metadata:
   labels:
     {{- include "labels.common" . | nindent 4}}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: capi.management-cluster.rules
   namespace: {{.Values.namespace}}
 spec:

diff --git a/...heus-rules/templates/kaas/turtles/alerting-rules/certificate.management-cluster.rules.yml b/...heus-rules/templates/kaas/turtles/alerting-rules/certificate.management-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: certificate.management-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/...etheus-rules/templates/kaas/turtles/alerting-rules/certificate.workload-cluster.rules.yml b/...etheus-rules/templates/kaas/turtles/alerting-rules/certificate.workload-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
-{{- end }}
+    {{- end }}
   name: certificate.workload-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/cluster-autoscaler.rules.yml b/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/cluster-autoscaler.rules.yml
@@ -5,9 +5,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
-{{- end }}
+    {{- end }}
   name: cluster-autoscaler.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/.../prometheus-rules/templates/kaas/turtles/alerting-rules/etcd.management-cluster.rules.yml b/.../prometheus-rules/templates/kaas/turtles/alerting-rules/etcd.management-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: etcd.management-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/etcd.workload-cluster.rules.yml b/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/etcd.workload-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
-{{- end }}
+    {{- end }}
   name: etcd.workload-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/etcdbackup.rules.yml b/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/etcdbackup.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: etcdbackup.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/inhibit.nodes.rules.yml b/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/inhibit.nodes.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: inhibit.nodes.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/management-cluster.rules.yml b/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/management-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: management-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/.../prometheus-rules/templates/kaas/turtles/alerting-rules/node.management-cluster.rules.yml b/.../prometheus-rules/templates/kaas/turtles/alerting-rules/node.management-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: node.management-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/node.workload-cluster.rules.yml b/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/node.workload-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
-{{- end }}
+    {{- end }}
   name: node.workload-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/pods.core.rules.yml b/helm/prometheus-rules/templates/kaas/turtles/alerting-rules/pods.core.rules.yml
@@ -4,7 +4,7 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-    {{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
     {{- end }}
   name: pods.core.rules

diff --git a/...ometheus-rules/templates/kaas/turtles/alerting-rules/storage.management-cluster.rules.yml b/...ometheus-rules/templates/kaas/turtles/alerting-rules/storage.management-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "management_cluster"
-{{- end }}
+    {{- end }}
   name: core.storage.management-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/...prometheus-rules/templates/kaas/turtles/alerting-rules/storage.workload-cluster.rules.yml b/...prometheus-rules/templates/kaas/turtles/alerting-rules/storage.workload-cluster.rules.yml
@@ -4,9 +4,9 @@ metadata:
   creationTimestamp: null
   labels:
     {{- include "labels.common" . | nindent 4 }}
-{{- if not .Values.mimir.enabled }}
+    {{- if eq .Values.managementCluster.provider.flavor "vintage" }}
     cluster_type: "workload_cluster"
-{{- end }}
+    {{- end }}
   name: core.storage.workload-cluster.rules
   namespace: {{ .Values.namespace  }}
 spec:

diff --git a/helm/prometheus-rules/templates/loki-ruler-datasource-configmap.yaml b/helm/prometheus-rules/templates/loki-ruler-datasource-configmap.yaml
@@ -1,4 +1,4 @@
-{{- if .Values.mimir.enabled }}
+{{- if eq .Values.managementCluster.provider.flavor "capi" }}
 apiVersion: v1
 kind: ConfigMap
 metadata: