Feature: Provision Individual Notification Policy Routes

[msvechla]

Is your feature request related to a problem? Please describe.

We need to dynamically add notification policy routes for different contact points.
In our case new Contact Points can be added dynamically during runtime and we need to be able to also dynamically add Notification Policy Routes to these Contact Points.

Describe the solution you'd like

Having a separate GrafanaNotificationPolicyRoute Kind and a routeSelector attribute in the existing GrafanaNotificationPolicy could probably solve this issue.

Something like:

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaNotificationPolicy
metadata:
  name: grafananotificationpolicy-sample
spec:
  instanceSelector:
    matchLabels:
      dashboards: "grafana"
  route:
    receiver: grafana-email-default
    group_by:
      - grafana_folder
      - alertname
    routeSelector:
      matchLabels:
        dynamicroute: "grafana"
    routes:
      - receiver: inline-from-notification-policy-a
        object_matchers:
          - - team
            - =
            - a
      - receiver: inline-from-notification-policy-b
        object_matchers:
          - - team
            - =
            - b

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaNotificationPolicyRoute
metadata:
  name: sample-route
  labels:
    dynamicroute: "grafana"
spec:
  instanceSelector:
    matchLabels:
      dashboards: "grafana"
  priority: 1
  routes:
    - receiver: grafana-receiver-1
       object_matchers:
          - - dynamic
            - =
            - 1

apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaNotificationPolicyRoute
metadata:
  name: sample-route
  labels:
    dynamicroute: "grafana"
spec:
  instanceSelector:
    matchLabels:
      dashboards: "grafana"
  priority: 2
  routes:
    - receiver: grafana-receiver-2
       object_matchers:
          - - dynamic
            - =
            - 2

These would result in the following merged routes:

    routes:
      - receiver: inline-from-notification-policy-a
        object_matchers:
          - - team
            - =
            - a
      - receiver: inline-from-notification-policy-b
        object_matchers:
          - - team
            - =
            - b
     - receiver: grafana-receiver-2
       object_matchers:
          - - dynamic
            - =
            - 2
    - receiver: grafana-receiver-1
       object_matchers:
          - - dynamic
            - =
            - 1

Assuming that routes with a higher priority value are added before lower priority values.

Please let me know what you think, this is just an initial idea from my side.

I would be open to help with the implementation on this.

Thanks a lot for your support!

[theSuess]

Thanks for the feature request. Having separate resources makes sense, but we worry that this would significantly complicate the operator code.

Your example is sensible, but we'd have to support nesting and composition at arbitrary layers.

Since this could become very technically involved, can you draft up a design proposal (similar to #1611)? The contents of this issue are a good starting point and with a PR open we can better discuss specific aspects

[msvechla]

Thanks for your input and help @theSuess . I opened a MR to add a proposal, so we can continue the discussion there: #1789