From cd5dac3d628c6d241094b634d07c8f23aba3dc05 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 25 Nov 2022 17:01:33 +0000 Subject: [PATCH] WIP: rewrite portion of Provider spec - More sections need to be moved from "working with" to "writing a ..." - When this is done, the whole needs to be read again and small adjustements are likely required to improve the flow of the doc. - Technical description of the behavior of the Provider types could be improved, this should be easier to do when everything has the same format. Signed-off-by: Hidde Beydals --- docs/spec/v1beta2/providers.md | 865 ++++++++++++++++++++------------- 1 file changed, 518 insertions(+), 347 deletions(-) diff --git a/docs/spec/v1beta2/providers.md b/docs/spec/v1beta2/providers.md index e23699050..8ee24e79b 100644 --- a/docs/spec/v1beta2/providers.md +++ b/docs/spec/v1beta2/providers.md @@ -4,7 +4,8 @@ The `Provider` API defines how events are encoded and where to send them. ## Example -The following is an example of how to send alerts to Slack when Flux fails to install or upgrade Flagger. +The following is an example of how to send alerts to Slack when Flux fails to +install or upgrade [Flagger](https://github.com/fluxcd/flagger). ```yaml --- @@ -48,7 +49,8 @@ In the above example: - The notification-controller starts listening for events sent for all HelmRepositories and HelmReleases in the `flagger-system` namespace. - When an event with severity `error` is received, the controller posts - a message on Slack containing the `summary` text and the Helm install or upgrade error. + a message on Slack containing the `summary` text and the Helm install or + upgrade error. - The controller uses the Slack Bot token from the secret indicated by the `Provider.spec.secretRef.name` to authenticate with the Slack API. @@ -90,6 +92,7 @@ You can run this example by saving the manifests into `slack-alerts.yaml`. As with all other Kubernetes config, a Provider needs `apiVersion`, `kind`, and `metadata` fields. The name of an Alert object must be a valid [DNS subdomain name](https://kubernetes.io/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). + A Provider also needs a [`.spec` section](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status). @@ -101,11 +104,10 @@ The supported alerting providers are: | Provider | Type | |---------------------------------------------------------|------------------| -| [Prometheus Alertmanager](#prometheus-alertmanager) | `alertmanager` | -| [Azure Event Hub](#azure-event-hub) | `azureeventhub` | -| [Discord](#discord) | `discord` | | [Generic webhook](#generic-webhook) | `generic` | | [Generic webhook with HMAC](#generic-webhook-with-hmac) | `generic-hmac` | +| [Azure Event Hub](#azure-event-hub) | `azureeventhub` | +| [Discord](#discord) | `discord` | | [GitHub dispatch](#github-dispatch) | `githubdispatch` | | [Google Chat](#google-chat) | `googlechat` | | [Grafana](#grafana) | `grafana` | @@ -113,6 +115,7 @@ The supported alerting providers are: | [Matrix](#matrix) | `matrix` | | [Microsoft Teams](#microsoft-teams) | `msteams` | | [Opsgenie](#opsgenie) | `opsgenie` | +| [Prometheus Alertmanager](#prometheus-alertmanager) | `alertmanager` | | [Rocket](#rocket) | `rocket` | | [Sentry](#sentry) | `sentry` | | [Slack](#slack) | `slack` | @@ -128,170 +131,15 @@ The supported providers for [Git commit status updates](#git-commit-status-updat | [GitHub](#github) | `github` | | [GitLab](#gitlab) | `gitlab` | -### Address - -`.spec.address` is an optional field that specifies the URL where the events are posted. - -If the address contains sensitive information such as tokens or passwords, it is -recommended to store the address in the Kubernetes secret referenced by `.spec.secretRef.name`. -When the referenced Secret contains an `address` key, the `.spec.address` value is ignored. - -### Channel - -`.spec.channel` is an optional field that specifies the channel where the events are posted. - -### Secret reference - -`.spec.secretRef.name` is an optional field to specify a name reference to a -Secret in the same namespace as the Provider, containing the authentication -credentials for the provider API. - -The Kubernetes secret can have any of the following keys: - -- `address` - overrides `.spec.address` -- `proxy` - overrides `.spec.proxy` -- `token` - used for authentication -- `headers` - HTTP headers values included in the POST request - -#### Address example - -For providers which embed tokens or other sensitive information in the URL, -the incoming webhooks address can be stored in the secret using the `address` key: - -```yaml ---- -apiVersion: v1 -kind: Secret -metadata: - name: my-provider-url - namespace: default -stringData: - address: "https://webhook.example.com/token" -``` - -#### Token example - -For providers which require token based authentication, the API token -can be stored in the secret using the `token` key: - -```yaml ---- -apiVersion: v1 -kind: Secret -metadata: - name: my-provider-auth - namespace: default -stringData: - token: "my-api-token" -``` - -#### HTTP headers example - -For providers which require specific HTTP headers to be attached to the POST request, -the headers can be set in the secret using the `headers` key: - -```yaml ---- -apiVersion: v1 -kind: Secret -metadata: - name: my-provider-headers - namespace: default -stringData: - headers: | - Authorization: my-api-token - X-Forwarded-Proto: https -``` - -#### Proxy auth example - -Some networks need to use an authenticated proxy to access external services. -Therefore, the proxy address can be stored as a secret to hide parameters like the username and password: - -```yaml ---- -apiVersion: v1 -kind: Secret -metadata: - name: my-provider-proxy - namespace: default -stringData: - proxy: "http://username:password@proxy_url:proxy_port" -``` - -### TLS certificates - -`.spec.certSecretRef` is an optional field to specify a name reference to a -Secret in the same namespace as the Provider, containing the TLS CA certificate. - -#### Example - -To enable notification-controller to communicate with a provider API over HTTPS -using a self-signed TLS certificate, set the `caFile` like so: - -```yaml ---- -apiVersion: notification.toolkit.fluxcd.io/v1beta2 -kind: Provider -metadata: - name: my-webhook - namespace: flagger-system -spec: - type: generic - address: https://my-webhook.internal - certSecretRef: - name: my-ca-crt ---- -apiVersion: v1 -kind: Secret -metadata: - name: my-ca-crt - namespace: default -stringData: - caFile: | - <--- CA Key ---> -``` - -### HTTP/S proxy +#### Alerting -`.spec.proxy` is an optional field to specify an HTTP/S proxy address. - -If the proxy address contains sensitive information such as basic auth credentials, it is -recommended to store the proxy in the Kubernetes secret referenced by `.spec.secretRef.name`. -When the referenced Secret contains a `proxy` key, the `.spec.proxy` value is ignored. +##### Generic webhook -### Interval +When `.spec.type` is set to `generic`, the controller will send an HTTP POST +request to the provided [Address](#address). -`.spec.interval` is a required field with a default of ten minutes that specifies -the time interval at which the controller reconciles the provider with its Secret -references. - -### Suspend - -`.spec.suspend` is an optional field to suspend the provider. -When set to `true`, the controller will stop sending events to this provider. -When the field is set to `false` or removed, it will resume. - -## Working with Providers - -### Generic webhook - -The `generic` webhook triggers an HTTP POST request to the provided endpoint. - -The `Gotk-Component` header identifies which component this event is coming -from, e.g. `source-controller`, `kustomize-controller`. - -``` -POST / HTTP/1.1 -Host: example.com -Accept-Encoding: gzip -Content-Length: 452 -Content-Type: application/json -Gotk-Component: source-controller -User-Agent: Go-http-client/1.1 -``` - -The body of the request looks like this: +The body of the request is a [JSON `Event` object](events.md#event-structure), +for example: ```json { @@ -314,59 +162,66 @@ The body of the request looks like this: } ``` -The `involvedObject` key contains the object that triggered the event. +Where the `involvedObject` key contains the metadata from the object triggering +the event. -You can add additional headers to the POST request by providing a `headers` field to the secret -referenced by the provider. An example is given below: +The controller includes a `Gotk-Component` header in the request, which can be +used to identify the component which sent the event, e.g. `source-controller` +or `notification-controller`. -```yaml -apiVersion: notification.toolkit.fluxcd.io/v1beta2 -kind: Provider -metadata: - name: generic - namespace: default -spec: - type: generic - address: https://api.github.com/repos/owner/repo/dispatches - secretRef: - name: generic-secret ---- -apiVersion: v1 -kind: Secret -metadata: - name: generic-secret - namespace: default -stringData: - headers: | - Authorization: token - X-Forwarded-Proto: https ``` +POST / HTTP/1.1 +Host: example.com +Accept-Encoding: gzip +Content-Length: 452 +Content-Type: application/json +Gotk-Component: kustomize-controller +User-Agent: Go-http-client/1.1 +``` + +You can add additional headers to the POST request using a [`headers` key in the +referenced Secret](#http-headers-example). -### Generic webhook with HMAC +##### Generic webhook with HMAC -If you set the `.spec.type` of a `Provider` resource to `generic-hmac` then the HTTP request -sent to the webhook will include the `X-Signature` HTTP header carrying the HMAC of the request body. -This allows the webhook server to authenticate the request. -The key used for the HMAC must be supplied in the `token` field of the Secret resource referenced in `.spec.secretRef`. -The HTTP header value has the following format: +When `.spec.type` is set to `generic-hmac`, the controller will send an HTTP +POST request to the provided [Address](#address) for an [Event](events.md#event-structure), +while including an `X-Signature` HTTP header carrying the HMAC of the request +body. The inclusion of the header allows the receiver to verify the +authenticity and integrity of the request. + +The `X-Signature` header is calculated by generating an HMAC of the request +body using the [`token` key from the referenced Secret](#token-example). The +HTTP header value has the following format: ``` -X-Signature: HASH_FUNC=HASH +X-Signature: = ``` -`HASH_FUNC` denotes the Hash function used to generate the HMAC and currently defaults -to `sha256` but may change in the future. You must make sure to take this value into -account when verifying the HMAC. `HASH` is the hex-encoded HMAC value. -The following Go code illustrates how the header is parsed and verified: +`` denotes the hash function used to generate the HMAC and +currently defaults to `sha256`, which may change in the future. `` is the +HMAC of the request body, encoded as a hexadecimal string. + +while `` is the hex-encoded HMAC value. + +The body of the request is a [JSON `Event` object](events.md#event-structure), +as described in the [Generic webhook](#generic-webhook) section. + +###### HMAC verification example + +The following example in Go shows how to verify the authenticity and integrity +of a request by using the X-Signature header. ```go -func verifySignature(sig string, payload, key []byte) error { - sigHdr := strings.Split(sig, "=") - if len(shgHdr) != 2 { +func verifySignature(signature string, payload, key []byte) error { + sig := strings.Split(signature, "=") + + if len(sig) != 2 { return fmt.Errorf("invalid signature value") } + var newF func() hash.Hash - switch sigHdr[0] { + switch sig[0] { case "sha224": newF = sha256.New224 case "sha256": @@ -378,38 +233,80 @@ func verifySignature(sig string, payload, key []byte) error { default: return fmt.Errorf("unsupported signature algorithm %q", sigHdr[0]) } + mac := hmac.New(newF, key) if _, err := mac.Write(payload); err != nil { - return fmt.Errorf("error MAC'ing payload: %w", err) + return fmt.Errorf("failed to write payload to HMAC encoder: %w", err) } + sum := fmt.Sprintf("%x", mac.Sum(nil)) - if sum != sigHdr[1] { - return fmt.Errorf("HMACs don't match: %#v != %#v", sum, sigHdr[1]) + if sum != sig[0] { + return fmt.Errorf("HMACs do not match: %#v != %#v", sum, sigHdr[0]) } return nil } -[...] -key := []byte("b1fad212fb1b87a56c79e5da48018650b85ab7cf") -if len(r.Header["X-Signature"]) > 0 { - if err := verifySignature(r.Header["X-Signature"][0], body, key); err != nil { - // handle signature verification failure here + +func handleRequest(w http.ResponseWriter, r *http.Request) { + // Require a X-Signature header + if len(r.Header["X-Signature"])) == 0 { + http.Error(w, "missing X-Signature header", http.StatusBadRequest) + return + } + + // Read the request body with a limit of 1MB + lr := io.LimitReader(r.Body, 1<<20) + body, err := ioutil.ReadAll(lr) + if err != nil { + http.Error(w, "failed to read request body", http.StatusBadRequest) + return } + + // Verify signature using the same token as the Secret referenced in + // Provider + key := "" + if err := verifySignature(r.Header.Get("X-Signature"), body, key); err != nil { + http.Error(w, fmt.Sprintf("failed to verify HMAC signature: %s", err.String()), http.StatusBadRequest) + return + } + + // Do something with the verified request body + // ... } ``` -### Slack +##### Slack -To send alerts to Slack, we recommend using a Slack Bot App token. -To obtain a token, please follow [Slack's guide on bot users](https://api.slack.com/bot-users). +When `.spec.type` is set to `slack`, the controller will send a message for an +[Event](events.md#event-structure) to the provided Slack API [Address](#address). -Once you have a Slack bot token (starts with `xoxb-`), create a secret for it with: +The Event will be formatted into a Slack message using an [Attachment](https://api.slack.com/reference/messaging/attachments), +with the metadata attached as fields, and the involved object as author. +The severity of the Event is used to set the color of the attachment. -```shell -kubectl create secret generic slack-token --from-literal=token=BOT-TOKEN -``` +When a [Channel](#channel) is provided, it will be added as a [`channel` +field](https://api.slack.com/methods/chat.postMessage#arg_channel) to the API +payload. Otherwise, the further configuration of the [Address](#address) will +determine the channel. + +When [Username](#username) is set, this will be added as a [`username` +field](https://api.slack.com/methods/chat.postMessage#arg_username) to the +payload, defaulting to the name of the reporting controller. + +This Provider type supports the configuration of a [proxy URL](#https-proxy) +and/or [TLS certificates](#tls-certificates). -Create a provider of type `slack`, with the address set to `https://slack.com/api/chat.postMessage` -and reference the `slack-token` secret: +###### Slack example + +To configure a Provider for Slack, we recommend using a Slack Bot App token which is +not attached to a specific Slack account. To obtain a token, please follow +[Slack's guide on creating an app](https://api.slack.com/authentication/basics#creating). + +Once you have obtained a token, [create a Secret containing the `token` +key](#token-example) and a `slack` Provider with the `address` set to +`https://slack.com/api/chat.postMessage`. + +Using this API endpoint, it is possible to send messages to multiple channels +by adding the integration to each channel. ```yaml --- @@ -424,9 +321,21 @@ spec: address: https://slack.com/api/chat.postMessage secretRef: name: slack-token +--- +apiVersion: v1 +kind: Secret +metadata: + name: slack-token + namespace: default +stringData: + token: xoxb-1234567890-1234567890-1234567890-1234567890 ``` -Slack legacy webhooks are also supported, the webhook URL can be set in the `address` field or in the secret: +###### Slack (legacy) example + +To configure a Provider for Slack using the [legacy incoming webhook API](https://api.slack.com/messaging/webhooks), +create a Secret with the `address` set to `https://hooks.slack.com/services/...`, +and a `slack` Provider with a [Secret reference](#address-example). ```yaml --- @@ -449,25 +358,26 @@ stringData: address: "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK" ``` -### Microsoft Teams +##### Microsoft Teams -To send alerts to Teams, first create an -[incoming webhook](https://docs.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook) -on the Microsoft Teams UI: +When `.spec.type` is set to `msteams`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Microsoft Teams [Address](#address). -1. Open the settings of the channel you want the notifications to be sent to. -2. Click on `Connectors`. -3. Click on the `Add` button for `Incoming Webhook`. -4. Click on `Configure` and copy the webhook URL given. +The Event will be formatted into a Microsoft Teams +[connector message](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/connectors-using#example-of-connector-message), +with the metadata attached as facts, and the involved object as summary. +The severity of the Event is used to set the color of the message. -Once you have the webhook URL, create a secret for it with: +This Provider type supports the configuration of a [proxy URL](#https-proxy) +and/or [TLS certificates](#tls-certificates), but lacks support for +configuring a [Channel](#channel). This can be configured during the +creation of the incoming webhook in Microsoft Teams. -```shell -kubectl create secret generic teams-webhook \ ---from-literal=address= -``` +###### Microsoft Teams example -Create a provider of type `msteam` and reference the `teams-webhook` secret: +To configure a Provider for Microsoft Teams, create a Secret with [the +`address`](#address-example) set to the [webhook URL](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook#create-incoming-webhooks-1), +and a `msteams` Provider with a [Secret reference](#address-example). ```yaml --- @@ -480,20 +390,34 @@ spec: type: msteams secretRef: name: slack-webhook +--- +apiVersion: v1 +kind: Secret +metadata: + name: msteams-webhook + namespace: default +stringData: + address: "https://xxx.webhook.office.com/..." ``` -### Discord +##### Discord -To send events to Discord, first [create a webhook](https://discord.com/developers/docs/resources/webhook#create-webhook). +When `.spec.type` is set to `discord`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Discord [Address](#address). -Once you have the webhook URL, create a secret for it with: +The Event will be formatted into a [Slack message](#slack) and send to the +`/slack` endpoint of the provided Discord [Address](#address). -```shell -kubectl create secret generic discord-webhook \ ---from-literal=address= -``` +This Provider type supports the configuration of a [proxy URL](#https-proxy) +and/or [TLS certificates](#tls-certificates), but lacks support for +configuring a [Channel](#channel). This can be configured [during the creation +of the address](https://discord.com/developers/docs/resources/webhook#create-webhook) + +###### Discord example -Create a provider of type `discord` and reference the `discord-webhook` secret: +To configure a Provider for Discord, create a Secret with [the `address`](#address-example) +set to the [webhook URL](https://discord.com/developers/docs/resources/webhook#create-webhook), +and a `discord` Provider with a [Secret reference](#secret-reference). ```yaml --- @@ -506,11 +430,40 @@ spec: type: discord secretRef: name: discord-webhook +--- +apiVersion: v1 +kind: Secret +metadata: + name: discord-webhook + namespace: default +stringData: + address: "https://discord.com/api/webhooks/..." ``` -### Sentry -To send events to Sentry, create a provider of type `sentry` and a secret with the Sentry URL: +##### Sentry + +When `.spec.type` is set to `sentry`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Sentry [Address](#address). + +Depending on the `severity` of the Event, the controller will capture a [Sentry +Event](https://develop.sentry.dev/sdk/event-payloads/)for `error`, or [Sentry +Transaction Event](https://develop.sentry.dev/sdk/event-payloads/transaction/) +with a [Span](https://develop.sentry.dev/sdk/event-payloads/span/) for `info`. +The metadata of the Event is included as [`extra` data](https://develop.sentry.dev/sdk/event-payloads/#optional-attributes) +in the Sentry Event, or as [Span `tags`](https://develop.sentry.dev/sdk/event-payloads/span/#attributes). + +The Provider's [Channel](#channel) is used to set the `environment` on the +Sentry client. + +This Provider type supports the configuration of +[TLS certificates](#tls-certificates). + +###### Sentry example + +To configure a Provider for Sentry, create a Secret with [the `address`](#address-example) +set to a [Sentry DSN](https://docs.sentry.io/product/sentry-basics/dsn-explainer/), +and a `sentry` Provider with a [Secret reference](#secret-reference). ```yaml --- @@ -534,24 +487,31 @@ stringData: address: "https://....@sentry.io/12341234" ``` -Note that the `.spec.channel` field can be used to specify which environment the messages are sent for. +**Note:** The `sentry` Provider also sends traces for events with the severity +`info`. This can be disabled by setting the `spec.eventSeverity` field to +`error` on an `Alert`. -The sentry provider also sends traces for events with the severity `info`. -This can be disabled by setting, the `Alert.spec.eventSeverity` field to `error`. +##### Telegram -### Telegram +When `.spec.type` is set to `telegram`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Telegram [Address](#address). -For telegram, You can get the token from [the botfather](https://core.telegram.org/bots#6-botfather) -and use `https://api.telegram.org/` as the address. +The Event will be formatted into a message string, with the metadata attached +as a list of key-value pairs. -Once you have a Telegram token, create a secret for it with: +The Provider's [Channel](#channel) is used to set the receiver of the message. +This can be a unique identifier (`-1234567890`) for the target chat, or +the username (`@username`) of the target channel. -```shell -kubectl create secret generic telegram-token \ ---from-literal=token=BOT-TOKEN -``` +This Provider type does not support the configuration of a [proxy URL](#https-proxy) +or [TLS certificates](#tls-certificates). + +###### Telegram example -Create a provider of type `telegram` and reference the `telegram-token` secret: +To configure a Provider for Telegram, create a Secret with [the `token`](#token-example) +obtained from [the BotFather](https://core.telegram.org/bots#how-do-i-create-a-bot), +and a `telegram` Provider with a [Secret reference](#secret-reference), and the +`address` set to `https://api.telegram.org`. ```yaml --- @@ -563,27 +523,31 @@ metadata: spec: type: telegram address: https://api.telegram.org - channel: "@fluxtest" # or "-1557265138" (channel id) + channel: "@fluxcd" # or "-1557265138" (channel id) secretRef: name: telegram-token ``` -Note that `.spec.channel` can be a unique identifier for the target chat -or the username of the target channel (in the format `@channelusername`). +##### Matrix -### Matrix +When `.spec.type` is set to `matrix`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Matrix [Address](#address). -For Matrix, the address is the homeserver URL and the token is the access token -returned by a call to `/login` or `/register`. +The Event will be formatted into a message string, with the metadata attached +as a list of key-value pairs, and send as a [`m.room.message` text event](https://spec.matrix.org/v1.3/client-server-api/#mroommessage) +to the provided Matrix [Address](#address). -Once you have a Matrix token, create a secret for it with: +The Provider's [Channel](#channel) is used to set the receiver of the message +using a room identifier (`!1234567890:example.org`). -```shell -kubectl create secret generic matrix-token \ ---from-literal=token=MY-TOKEN -``` +This provider type does support the configuration of [TLS +certificates](#tls-certificates). -Create a provider of type `matrix` and reference the `matrix-token` secret: +###### Matrix example + +To configure a Provider for Matrix, create a Secret with [the `token`](#token-example) +obtained from [the Matrix endpoint](https://matrix.org/docs/guides/client-server-api#registration), +and a `matrix` Provider with a [Secret reference](#secret-reference). ```yaml apiVersion: notification.toolkit.fluxcd.io/v1beta2 @@ -599,20 +563,22 @@ spec: name: matrix-token ``` -Note that `.spec.channel` holds the room ID. +##### Lark -### Lark +When `.spec.type` is set to `lark`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Lark [Address](#address). -For sending notifications to Lark, you will have to -[add a bot to the group](https://www.larksuite.com/hc/en-US/articles/360048487736-Bot-Use-bots-in-groups#III.%20How%20to%20configure%20custom%20bots%20in%20a%20group%C2%A0) -and set up a webhook for that bot account. This serves as the address field in the secret: +The Event will be formatted into a [Lark Message card](https://open.larksuite.com/document/ukTMukTMukTM/uczM3QjL3MzN04yNzcDN), +with the metadata written to the message string. -```shell -kubectl create secret generic lark-webhook \ ---from-literal=address= -``` +This Provider type does not support the configuration of a [proxy URL](#https-proxy) +or [TLS certificates](#tls-certificates). + +###### Lark example -Create a provider of type `lark` and reference the `lark-webhook` secret: +To configure a Provider for Lark, create a Secret with [the `address`](#address-example) +obtained from [adding a bot to a group](https://open.larksuite.com/document/uAjLw4CM/ukTMukTMukTM/bot-v3/use-custom-bots-in-a-group#57181e84), +and a `lark` Provider with a [Secret reference](#secret-reference). ```yaml apiVersion: notification.toolkit.fluxcd.io/v1beta2 @@ -624,20 +590,32 @@ spec: type: lark secretRef: name: lark-webhook +--- +apiVersion: v1 +kind: Secret +metadata: + name: lark-webhook + namespace: default +stringData: + address: "https://open.larksuite.com/open-apis/bot/v2/hook/xxxxxxxxxxxxxxxxx" ``` -### Rocket +##### Rocket -To send events to Rocket chat, first [create an incoming webhook](https://docs.rocket.chat/guides/administration/admin-panel/integrations#create-a-new-incoming-webhook). +When `.spec.type` is set to `rocket`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Rocket [Address](#address). -Once you have the webhook URL, create a secret for it with: +The Event will be formatted into a [Slack message](#slack) and send as a +payload the provided Rocket [Address](#address). -```shell -kubectl create secret generic rocket-webhook \ ---from-literal=address= -``` +This Provider type does support the configuration of a [proxy URL](#https-proxy) +and [TLS certificates](#tls-certificates). -Create a provider of type `rocket` and reference the `rocket-webhook` secret: +###### Rocket example + +To configure a Provider for Rocket, create a Secret with [the `address`](#address-example) +set to the Rocket [webhook URL](https://docs.rocket.chat/guides/administration/admin-panel/integrations#incoming-webhook-script), +and a `rocket` Provider with a [Secret reference](#secret-reference). ```yaml --- @@ -652,18 +630,22 @@ spec: name: rocket-webhook ``` -### Google Chat +##### Google Chat -To send notifications to Google chat, first [create an incoming webhook](https://developers.google.com/chat/how-tos/webhooks#create_a_webhook). +When `.spec.type` is set to `googlechat`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Google Chat [Address](#address). -Once you have the webhook URL, create a secret for it with: +The Event will be formatted into a [Google Chat card message](https://developers.google.com/chat/api/reference/rest/v1/cards-v1), +with the metadata added as a list of [key-value pairs](https://developers.google.com/chat/api/reference/rest/v1/cards-v1#keyvalue) +in a [widget](https://developers.google.com/chat/api/reference/rest/v1/cards-v1#widgetmarkup). -```shell -kubectl create secret generic google-webhook \ ---from-literal=address= -``` +This Provider type does support the configuration of a [proxy URL](#https-proxy). + +###### Google Chat example -Create a provider of type `googlechat` and reference the `google-webhook` secret: +To configure a Provider for Google Chat, create a Secret with [the `address`](#address-example) +set to the Google Chat [webhook URL](https://developers.google.com/chat/how-tos/webhooks#create_a_webhook), +and a `googlechat` Provider with a [Secret reference](#secret-reference). ```yaml --- @@ -676,23 +658,37 @@ spec: type: googlechat secretRef: name: google-webhook +--- +apiVersion: v1 +kind: Secret +metadata: + name: google-webhook + namespace: default +stringData: + address: https://chat.googleapis.com/v1/spaces/... ``` -### Opsgenie +##### Opsgenie -To send notifications to Opsgenie, first -[add a REST API integration](https://support.atlassian.com/opsgenie/docs/create-a-default-api-integration/). +When `.spec.type` is set to `opsgenie`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Opsgenie [Address](#address). -Once you have a Opsgenie API key, create a secret for it with: +The Event will be formatted into a [Opsgenie alert](https://docs.opsgenie.com/docs/alert-api#section-create-alert-request), +with the metadata added to the [`details` field](https://docs.opsgenie.com/docs/alert-api#create-alert) +as a list of key-value pairs. -```shell -kubectl create secret generic opsgenie-token \ ---from-literal=token= -``` +This Provider type does support the configuration of a [proxy URL](#https-proxy) +and [TLS certificates](#tls-certificates). + +###### Opsgenie example -Create a provider of type `opsgenie` and reference the `opsgenie-token` secret: +To configure a Provider for Opsgenie, create a Secret with [the `token`](#token-example) +set to the Opsgenie [API token](https://support.atlassian.com/opsgenie/docs/create-a-default-api-integration/), +and a `opsgenie` Provider with a [Secret reference](#secret-reference) and the +`address` set to `https://api.opsgenie.com/v2/alerts`. ```yaml +--- apiVersion: notification.toolkit.fluxcd.io/v1beta2 kind: Provider metadata: @@ -703,34 +699,28 @@ spec: address: https://api.opsgenie.com/v2/alerts secretRef: name: opsgenie-token -``` - -### Prometheus Alertmanager - -To send events to the Prometheus [Alertmanager v2 API](https://github.com/prometheus/alertmanager/blob/main/api/v2/openapi.yaml), -create a provider of type `alertmanager` and set the `address` field to the `api/v2/alerts` endpoint: - -```yaml -apiVersion: notification.toolkit.fluxcd.io/v1beta2 -kind: Provider +--- +apiVersion: v1 +kind: Secret metadata: - name: alertmanager + name: opsgenie-token namespace: default -spec: - type: alertmanager - # webhook address (ignored if secretRef is specified) - address: https://....@/api/v2/alerts/" +stringData: + token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ``` -If Alertmanager has basic authentication configured, it is recommended to use -`.spec.secretRef` and include the `username:password` in the address string inside the secret. +##### Prometheus Alertmanager -When an event is received, the controller will send a single alert with at least one annotation -which is the `message` found for the event. -If an `Alert.spec.summary` is provided, an additional "summary" annotation will be added. +When `.spec.type` is set to `alertmanager`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Prometheus Alertmanager +[Address](#address). -The provider will send the following labels for the event: +The Event will be formatted into a `firing` [Prometheus Alertmanager +alert](https://prometheus.io/docs/alerting/latest/notifications/#alert), +with the metadata added to the `labels` fields, and the `message` (and optional +`.metadata.summary`) added as annotations. +In addition to the metadata from the Event, the following labels will be added: | Label | Description | |-----------|------------------------------------------------------------------------------------------------------| @@ -742,28 +732,65 @@ The provider will send the following labels for the event: | name | The name of the involved object associated with the event | | namespace | The namespace of the involved object associated with the event | -### Webex +This Provider type does support the configuration of a [proxy URL](#https-proxy) +and [TLS certificates](#tls-certificates). -General steps on how to send notifications to a Webex space: +###### Prometheus Alertmanager example -From the Webex App UI: +To configure a Provider for Prometheus Alertmanager, create a Secret with [the +`address`](#address-example) set to the Prometheus Alertmanager [HTTP API +URL](https://prometheus.io/docs/alerting/latest/https/#http-traffic) +including Basic Auth credentials, and a `alertmanager` Provider with a [Secret +reference](#secret-reference). -- create a Webex space where you want notifications to be sent -- after creating a Webex bot (described in next section), add the bot email address to the Webex space ("People | Add people") +```yaml +--- +apiVersion: notification.toolkit.fluxcd.io/v1beta2 +kind: Provider +metadata: + name: alertmanager + namespace: default +spec: + type: alertmanager + secretRef: + name: alertmanager-address +--- +apiVersion: v1 +kind: Secret +metadata: + name: alertmanager-address + namespace: default +stringData: + address: https://username:password@/api/v2/alerts/" +``` + +##### Webex + +When `.spec.type` is set to `webex`, the controller will send a payload for +an [Event](events.md#event-structure) to the provided Webex [Address](#address). + +The Event will be formatted into a message string, with the metadata attached +as a list of key-value pairs, and send as a [Webex message](https://developer.webex.com/docs/api/v1/messages/create-a-message). + +The [Channel](#channel) is used to set the ID of the room to send the message +to. -Register to https://developer.webex.com/, after signing in: +This Provider type does support the configuration of a [proxy URL](#https-proxy) +and [TLS certificates](#tls-certificates). -- Create a bot for forwarding Flux notifications to a Webex Space - (User profile icon | MyWebexApps | Create a New App | Create a Bot). -- Make a note of the bot email address, this email needs to be added to the Webex space from the Webex App. -- Generate a bot access token, this is the ID to use in the kubernetes Secret "token" field. -- Find the room ID associated to the webex space using https://developer.webex.com/docs/api/v1/rooms/list-rooms - (select GET, click on "Try It" and search the GET results for the matching Webex space entry), - this is the ID to use in the webex Provider manifest "channel" field. +###### Webex example -Example: +To configure a Provider for Webex, create a Secret with [the `token`](#token-example) +set to the Webex [access token](https://developer.webex.com/docs/api/getting-started#authentication), +and a `webex` Provider with a [Secret reference](#secret-reference) and the +`address` set to `https://webexapis.com/v1/messages`. + +**Note:** To be able to send messages to a Webex room, the bot needs to be +added to the room. Failing to do so will result in 404 errors, logged by the +controller. ```yaml +--- apiVersion: notification.toolkit.fluxcd.io/v1beta2 kind: Provider metadata: @@ -785,14 +812,158 @@ stringData: token: ``` -Notes: +### Address + +`.spec.address` is an optional field that specifies the URL where the events are posted. + +If the address contains sensitive information such as tokens or passwords, it is +recommended to store the address in the Kubernetes secret referenced by `.spec.secretRef.name`. +When the referenced Secret contains an `address` key, the `.spec.address` value is ignored. + +### Channel + +`.spec.channel` is an optional field that specifies the channel where the events are posted. + +### Username + +`.spec.username` is an optional field that specifies the username used to post +the events. Can be overwritten with a [Secret reference](#secret-reference). + +### Secret reference + +`.spec.secretRef.name` is an optional field to specify a name reference to a +Secret in the same namespace as the Provider, containing the authentication +credentials for the provider API. + +The Kubernetes secret can have any of the following keys: + +- `address` - overrides `.spec.address` +- `proxy` - overrides `.spec.proxy` +- `token` - used for authentication +- `username` - overrides `.spec.username` +- `headers` - HTTP headers values included in the POST request + +#### Address example + +For providers which embed tokens or other sensitive information in the URL, +the incoming webhooks address can be stored in the secret using the `address` key: + +```yaml +--- +apiVersion: v1 +kind: Secret +metadata: + name: my-provider-url + namespace: default +stringData: + address: "https://webhook.example.com/token" +``` + +#### Token example -- `.spec.address` should always be set to the same global Webex API gateway `https://webexapis.com/v1/messages` -- `.spec.channel` should contain the Webex space room ID as obtained from `https://developer.webex.com/` (long alphanumeric string copied as is). +For providers which require token based authentication, the API token +can be stored in the secret using the `token` key: + +```yaml +--- +apiVersion: v1 +kind: Secret +metadata: + name: my-provider-auth + namespace: default +stringData: + token: "my-api-token" +``` + +#### HTTP headers example + +For providers which require specific HTTP headers to be attached to the POST request, +the headers can be set in the secret using the `headers` key: + +```yaml +--- +apiVersion: v1 +kind: Secret +metadata: + name: my-provider-headers + namespace: default +stringData: + headers: | + Authorization: my-api-token + X-Forwarded-Proto: https +``` + +#### Proxy auth example + +Some networks need to use an authenticated proxy to access external services. +Therefore, the proxy address can be stored as a secret to hide parameters like the username and password: + +```yaml +--- +apiVersion: v1 +kind: Secret +metadata: + name: my-provider-proxy + namespace: default +stringData: + proxy: "http://username:password@proxy_url:proxy_port" +``` + +### TLS certificates + +`.spec.certSecretRef` is an optional field to specify a name reference to a +Secret in the same namespace as the Provider, containing the TLS CA certificate. + +#### Example + +To enable notification-controller to communicate with a provider API over HTTPS +using a self-signed TLS certificate, set the `caFile` like so: + +```yaml +--- +apiVersion: notification.toolkit.fluxcd.io/v1beta2 +kind: Provider +metadata: + name: my-webhook + namespace: flagger-system +spec: + type: generic + address: https://my-webhook.internal + certSecretRef: + name: my-ca-crt +--- +apiVersion: v1 +kind: Secret +metadata: + name: my-ca-crt + namespace: default +stringData: + caFile: | + <--- CA Key ---> +``` + +### HTTP/S proxy + +`.spec.proxy` is an optional field to specify an HTTP/S proxy address. + +If the proxy address contains sensitive information such as basic auth credentials, it is +recommended to store the proxy in the Kubernetes secret referenced by `.spec.secretRef.name`. +When the referenced Secret contains a `proxy` key, the `.spec.proxy` value is ignored. + +### Interval + +`.spec.interval` is a required field with a default of ten minutes that specifies +the time interval at which the controller reconciles the provider with its Secret +references. + +### Suspend + +`.spec.suspend` is an optional field to suspend the provider. +When set to `true`, the controller will stop sending events to this provider. +When the field is set to `false` or removed, it will resume. + +## Working with Providers -If you do not see any notifications in the targeted Webex space, check that you have added the bot -email address to the Webex space, if the bot email address is not added to the space, -the notification-controller will log a 404 room not found error every time a notification is sent out. ### Grafana