Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(alerting): Add GitLab alerting provider #485

Merged
merged 9 commits into from
May 31, 2023
Binary file added .github/assets/gitlab-alerts.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Have any feedback or questions? [Create a discussion](https://github.com/TwiN/ga
- [Configuring Discord alerts](#configuring-discord-alerts)
- [Configuring Email alerts](#configuring-email-alerts)
- [Configuring GitHub alerts](#configuring-github-alerts)
- [Configuring GitLab alerts](#configuring-gitlab-alerts)
- [Configuring Google Chat alerts](#configuring-google-chat-alerts)
- [Configuring Matrix alerts](#configuring-matrix-alerts)
- [Configuring Mattermost alerts](#configuring-mattermost-alerts)
Expand Down Expand Up @@ -411,6 +412,7 @@ ignored.
| `alerting.discord` | Configuration for alerts of type `discord`. <br />See [Configuring Discord alerts](#configuring-discord-alerts). | `{}` |
| `alerting.email` | Configuration for alerts of type `email`. <br />See [Configuring Email alerts](#configuring-email-alerts). | `{}` |
| `alerting.github` | Configuration for alerts of type `github`. <br />See [Configuring GitHub alerts](#configuring-github-alerts). | `{}` |
| `alerting.gitlab` | Configuration for alerts of type `gitlab`. <br />See [Configuring GitLab alerts](#configuring-gitlab-alerts). | `{}` |
| `alerting.googlechat` | Configuration for alerts of type `googlechat`. <br />See [Configuring Google Chat alerts](#configuring-google-chat-alerts). | `{}` |
| `alerting.matrix` | Configuration for alerts of type `matrix`. <br />See [Configuring Matrix alerts](#configuring-matrix-alerts). | `{}` |
| `alerting.mattermost` | Configuration for alerts of type `mattermost`. <br />See [Configuring Mattermost alerts](#configuring-mattermost-alerts). | `{}` |
Expand Down Expand Up @@ -550,6 +552,47 @@ endpoints:

![GitHub alert](.github/assets/github-alerts.png)

#### Configuring GitLab alerts
| Parameter | Description | Default |
|:------------------------------------|:----------------------------------------------------------------------------------------------------------------|:--------------|
| `alerting.gitlab` | Configuration for alerts of type `gitlab` | `{}` |
| `alerting.gitlab.webhook-url` | GitLab alert webhook URL (e.g. `https://gitlab.com/hlidotbe/example/alerts/notify/gatus/xxxxxxxxxxxxxxxx.json`) | Required `""` |
| `alerting.gitlab.authorization-key` | Personal access token to use for authentication. <br />Must have at least RW on issues and RO on metadata. | Required `""` |
| `alerting.gitlab.severity` | Override default severity (critical), can be one of `critical, high, medium, low, info, unknown` | `""` |
| `alerting.gitlab.monitoring-tool` | Override the monitoring tool name (gatus) | `"gatus"` |
| `alerting.gitlab.environment-name` | Set gitlab environment's name. Required to display alerts on a dashboard. | `""` |
| `alerting.gitlab.service` | Override endpoint displayname | `""` |
| `alerting.gitlab.default-alert` | Default alert configuration. <br />See [Setting a default alert](#setting-a-default-alert). | N/A |

The GitLab alerting provider creates an alert prefixed with `🚨 (gatus):` and suffixed with the endpoint's display
name for each alert. If `send-on-resolved` is set to `true` on the endpoint alert, the alert will be automatically
closed when the alert is resolved. See
https://docs.gitlab.com/ee/operations/incident_management/integrations.html#configuration to configure the endpoint.

```yaml
alerting:
gitlab:
webhook-url: "https://gitlab.com/hlidotbe/example/alerts/notify/gatus/xxxxxxxxxxxxxxxx.json"
authorization-key: "12345"

endpoints:
- name: example
url: "https://twin.sh/health"
interval: 5m
conditions:
- "[STATUS] == 200"
- "[BODY].status == UP"
- "[RESPONSE_TIME] < 75"
alerts:
- type: gitlab
failure-threshold: 2
success-threshold: 3
send-on-resolved: true
description: "Everything's burning AAAAAHHHHHHHHHHHHHHH"
```

![GitLab alert](.github/assets/gitlab-alerts.png)


#### Configuring Google Chat alerts
| Parameter | Description | Default |
Expand Down
3 changes: 3 additions & 0 deletions alerting/alert/type.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ const (
// TypeGitHub is the Type for the github alerting provider
TypeGitHub Type = "github"

// TypeGitLab is the Type for the gitlab alerting provider
TypeGitLab Type = "gitlab"

// TypeGoogleChat is the Type for the googlechat alerting provider
TypeGoogleChat Type = "googlechat"

Expand Down
4 changes: 4 additions & 0 deletions alerting/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"github.com/TwiN/gatus/v5/alerting/provider/discord"
"github.com/TwiN/gatus/v5/alerting/provider/email"
"github.com/TwiN/gatus/v5/alerting/provider/github"
"github.com/TwiN/gatus/v5/alerting/provider/gitlab"
"github.com/TwiN/gatus/v5/alerting/provider/googlechat"
"github.com/TwiN/gatus/v5/alerting/provider/matrix"
"github.com/TwiN/gatus/v5/alerting/provider/mattermost"
Expand Down Expand Up @@ -39,6 +40,9 @@ type Config struct {
// GitHub is the configuration for the github alerting provider
GitHub *github.AlertProvider `yaml:"github,omitempty"`

// GitLab is the configuration for the gitlab alerting provider
GitLab *gitlab.AlertProvider `yaml:"gitlab,omitempty"`

// GoogleChat is the configuration for the googlechat alerting provider
GoogleChat *googlechat.AlertProvider `yaml:"googlechat,omitempty"`

Expand Down
147 changes: 147 additions & 0 deletions alerting/provider/gitlab/gitlab.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
package gitlab

import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"time"

"github.com/TwiN/gatus/v5/alerting/alert"
"github.com/TwiN/gatus/v5/client"
"github.com/TwiN/gatus/v5/core"
"github.com/google/uuid"
)

// AlertProvider is the configuration necessary for sending an alert using GitLab
type AlertProvider struct {
WebhookURL string `yaml:"webhook-url"` // The webhook url provided by GitLab
AuthorizationKey string `yaml:"authorization-key"` // The authorization key provided by GitLab

// DefaultAlert is the default alert configuration to use for endpoints with an alert of the appropriate type
DefaultAlert *alert.Alert `yaml:"default-alert,omitempty"`

// Severity can be one of: critical, high, medium, low, info, unknown. Defaults to critical
Severity string `yaml:"severity,omitempty"`
// MonitoringTool overrides the name sent to gitlab. Defaults to gatus
MonitoringTool string `yaml:"monitoring-tool,omitempty"`
// EnvironmentName is the name of the associated GitLab environment. Required to display alerts on a dashboard.
EnvironmentName string `yaml:"environment-name,omitempty"`
// Service affected. Defaults to endpoint display name
Service string `yaml:"service,omitempty"`
}

// IsValid returns whether the provider's configuration is valid
func (provider *AlertProvider) IsValid() bool {
if len(provider.AuthorizationKey) == 0 || len(provider.WebhookURL) == 0 {
return false
}
// Validate format of the repository URL
_, err := url.Parse(provider.WebhookURL)
if err != nil {
return false
}
return true
}

// Send creates an issue in the designed RepositoryURL if the resolved parameter passed is false,
// or closes the relevant issue(s) if the resolved parameter passed is true.
func (provider *AlertProvider) Send(endpoint *core.Endpoint, alert *alert.Alert, result *core.Result, resolved bool) error {
if len(alert.ResolveKey) == 0 {
alert.ResolveKey = uuid.NewString()
}

buffer := bytes.NewBuffer(provider.buildAlertBody(endpoint, alert, result, resolved))
request, err := http.NewRequest(http.MethodPost, provider.WebhookURL, buffer)
if err != nil {
return err
}
request.Header.Set("Content-Type", "application/json")
request.Header.Set("Authorization", fmt.Sprintf("Bearer %s", provider.AuthorizationKey))
response, err := client.GetHTTPClient(nil).Do(request)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode > 399 {
body, _ := io.ReadAll(response.Body)
return fmt.Errorf("call to provider alert returned status code %d: %s", response.StatusCode, string(body))
}
return err
}

type AlertBody struct {
Title string `json:"title,omitempty"` // The title of the alert.
Description string `json:"description,omitempty"` // A high-level summary of the problem.
StartTime string `json:"start_time,omitempty"` // The time of the alert. If none is provided, a current time is used.
EndTime string `json:"end_time,omitempty"` // The resolution time of the alert. If provided, the alert is resolved.
Service string `json:"service,omitempty"` // The affected service.
MonitoringTool string `json:"monitoring_tool,omitempty"` // The name of the associated monitoring tool.
Hosts string `json:"hosts,omitempty"` // One or more hosts, as to where this incident occurred.
Severity string `json:"severity,omitempty"` // The severity of the alert. Case-insensitive. Can be one of: critical, high, medium, low, info, unknown. Defaults to critical if missing or value is not in this list.
Fingerprint string `json:"fingerprint,omitempty"` // The unique identifier of the alert. This can be used to group occurrences of the same alert.
GitlabEnvironmentName string `json:"gitlab_environment_name,omitempty"` // The name of the associated GitLab environment. Required to display alerts on a dashboard.
}

func (provider *AlertProvider) monitoringTool() string {
if len(provider.MonitoringTool) > 0 {
return provider.MonitoringTool
}
return "gatus"
}

func (provider *AlertProvider) service(endpoint *core.Endpoint) string {
if len(provider.Service) > 0 {
return provider.Service
}
return endpoint.DisplayName()
}

// buildAlertBody builds the body of the alert
func (provider *AlertProvider) buildAlertBody(endpoint *core.Endpoint, alert *alert.Alert, result *core.Result, resolved bool) []byte {
body := AlertBody{
Title: fmt.Sprintf("alert(%s): %s", provider.monitoringTool(), provider.service(endpoint)),
StartTime: result.Timestamp.Format(time.RFC3339),
Service: provider.service(endpoint),
MonitoringTool: provider.monitoringTool(),
Hosts: endpoint.URL,
GitlabEnvironmentName: provider.EnvironmentName,
Severity: provider.Severity,
Fingerprint: alert.ResolveKey,
}
if resolved {
body.EndTime = result.Timestamp.Format(time.RFC3339)
}

var results string
for _, conditionResult := range result.ConditionResults {
var prefix string
if conditionResult.Success {
prefix = ":white_check_mark:"
} else {
prefix = ":x:"
}
results += fmt.Sprintf("- %s - `%s`\n", prefix, conditionResult.Condition)
}
var description string
if alertDescription := alert.GetDescription(); len(alertDescription) > 0 {
description = ":\n> " + alertDescription
}
var message string
if resolved {
message = fmt.Sprintf("An alert for *%s* has been resolved after passing successfully %d time(s) in a row", endpoint.DisplayName(), alert.SuccessThreshold)
} else {
message = fmt.Sprintf("An alert for *%s* has been triggered due to having failed %d time(s) in a row", endpoint.DisplayName(), alert.FailureThreshold)
}
body.Description = message + description + "\n\n## Condition results\n" + results

json, _ := json.Marshal(body)
return json
}

// GetDefaultAlert returns the provider's default alert configuration
func (provider AlertProvider) GetDefaultAlert() *alert.Alert {
return provider.DefaultAlert
}
158 changes: 158 additions & 0 deletions alerting/provider/gitlab/gitlab_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
package gitlab

import (
"net/http"
"strings"
"testing"

"github.com/TwiN/gatus/v5/alerting/alert"
"github.com/TwiN/gatus/v5/client"
"github.com/TwiN/gatus/v5/core"
"github.com/TwiN/gatus/v5/test"
)

func TestAlertDefaultProvider_IsValid(t *testing.T) {
scenarios := []struct {
Name string
Provider AlertProvider
Expected bool
}{
{
Name: "invalid",
Provider: AlertProvider{WebhookURL: "", AuthorizationKey: ""},
Expected: false,
},
{
Name: "missing-webhook-url",
Provider: AlertProvider{WebhookURL: "", AuthorizationKey: "12345"},
Expected: false,
},
{
Name: "missing-authorization-key",
Provider: AlertProvider{WebhookURL: "https://gitlab.com/hlidotbe/text/alerts/notify/gatus/xxxxxxxxxxxxxxxx.json", AuthorizationKey: ""},
Expected: false,
},
{
Name: "invalid-url",
Provider: AlertProvider{WebhookURL: " http://foo.com", AuthorizationKey: "12345"},
Expected: false,
},
}
for _, scenario := range scenarios {
t.Run(scenario.Name, func(t *testing.T) {
if scenario.Provider.IsValid() != scenario.Expected {
t.Errorf("expected %t, got %t", scenario.Expected, scenario.Provider.IsValid())
}
})
}
}

func TestAlertProvider_Send(t *testing.T) {
defer client.InjectHTTPClient(nil)
firstDescription := "description-1"
secondDescription := "description-2"
scenarios := []struct {
Name string
Provider AlertProvider
Alert alert.Alert
Resolved bool
MockRoundTripper test.MockRoundTripper
ExpectedError bool
}{
{
Name: "triggered-error",
Provider: AlertProvider{WebhookURL: "https://gitlab.com/hlidotbe/text/alerts/notify/gatus/xxxxxxxxxxxxxxxx.json", AuthorizationKey: "12345"},
Alert: alert.Alert{Description: &firstDescription, SuccessThreshold: 5, FailureThreshold: 3},
Resolved: false,
ExpectedError: false,
MockRoundTripper: test.MockRoundTripper(func(r *http.Request) *http.Response {
return &http.Response{StatusCode: http.StatusOK, Body: http.NoBody}
}),
},
{
Name: "resolved-error",
Provider: AlertProvider{WebhookURL: "https://gitlab.com/hlidotbe/text/alerts/notify/gatus/xxxxxxxxxxxxxxxx.json", AuthorizationKey: "12345"},
Alert: alert.Alert{Description: &secondDescription, SuccessThreshold: 5, FailureThreshold: 3},
Resolved: true,
ExpectedError: false,
MockRoundTripper: test.MockRoundTripper(func(r *http.Request) *http.Response {
return &http.Response{StatusCode: http.StatusOK, Body: http.NoBody}
}),
},
}
for _, scenario := range scenarios {
t.Run(scenario.Name, func(t *testing.T) {
client.InjectHTTPClient(&http.Client{Transport: scenario.MockRoundTripper})
err := scenario.Provider.Send(
&core.Endpoint{Name: "endpoint-name", Group: "endpoint-group"},
&scenario.Alert,
&core.Result{
ConditionResults: []*core.ConditionResult{
{Condition: "[CONNECTED] == true", Success: scenario.Resolved},
{Condition: "[STATUS] == 200", Success: scenario.Resolved},
},
},
scenario.Resolved,
)
if scenario.ExpectedError && err == nil {
t.Error("expected error, got none")
}
if !scenario.ExpectedError && err != nil {
t.Error("expected no error, got", err.Error())
}
})
}
}

func TestAlertProvider_buildAlertBody(t *testing.T) {
firstDescription := "description-1"
scenarios := []struct {
Name string
Endpoint core.Endpoint
Provider AlertProvider
Alert alert.Alert
ExpectedBody string
}{
{
Name: "triggered",
Endpoint: core.Endpoint{Name: "endpoint-name", URL: "https://example.org"},
Provider: AlertProvider{},
Alert: alert.Alert{Description: &firstDescription, FailureThreshold: 3},
ExpectedBody: "{\"title\":\"🚨 (gatus): endpoint-name\",\"description\":\"An alert for *endpoint-name* has been triggered due to having failed 3 time(s) in a row:\\n\\u003e description-1\\n\\n## Condition results\\n- :white_check_mark: - `[CONNECTED] == true`\\n- :x: - `[STATUS] == 200`\\n\",\"start_time\":\"0001-01-01T00:00:00Z\",\"service\":\"endpoint-name\",\"monitoring_tool\":\"gatus\",\"hosts\":\"https://example.org\"}",
},
{
Name: "no-description",
Endpoint: core.Endpoint{Name: "endpoint-name", URL: "https://example.org"},
Provider: AlertProvider{},
Alert: alert.Alert{FailureThreshold: 10},
ExpectedBody: "{\"title\":\"🚨 (gatus): endpoint-name\",\"description\":\"An alert for *endpoint-name* has been triggered due to having failed 10 time(s) in a row\\n\\n## Condition results\\n- :white_check_mark: - `[CONNECTED] == true`\\n- :x: - `[STATUS] == 200`\\n\",\"start_time\":\"0001-01-01T00:00:00Z\",\"service\":\"endpoint-name\",\"monitoring_tool\":\"gatus\",\"hosts\":\"https://example.org\"}",
TwiN marked this conversation as resolved.
Show resolved Hide resolved
},
}
for _, scenario := range scenarios {
t.Run(scenario.Name, func(t *testing.T) {
body := scenario.Provider.buildAlertBody(
&scenario.Endpoint,
&scenario.Alert,
&core.Result{
ConditionResults: []*core.ConditionResult{
{Condition: "[CONNECTED] == true", Success: true},
{Condition: "[STATUS] == 200", Success: false},
},
},
false,
)
if strings.TrimSpace(string(body)) != strings.TrimSpace(scenario.ExpectedBody) {
t.Errorf("expected:\n%s\ngot:\n%s", scenario.ExpectedBody, body)
}
})
}
}

func TestAlertProvider_GetDefaultAlert(t *testing.T) {
if (AlertProvider{DefaultAlert: &alert.Alert{}}).GetDefaultAlert() == nil {
t.Error("expected default alert to be not nil")
}
if (AlertProvider{DefaultAlert: nil}).GetDefaultAlert() != nil {
t.Error("expected default alert to be nil")
}
}
Loading