diff --git a/.gitignore b/.gitignore index dfe8f6ee0..4ca49f2dc 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,5 @@ # Binary -./dynatrace-service +dynatrace-service # Auto-generated files ./deploy/manifests/dynatrace/gen/cr* @@ -11,5 +11,8 @@ creds_dt.json vendor/* +# local files +_* + # GoLand IDE -.idea \ No newline at end of file +.idea diff --git a/README.md b/README.md index bcdfa3344..a2a4ff391 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,147 @@ -# Dynatrace Service and Dynatrace OneAgent Operator +# (grabnerandi in dev) Dynatrace Service + +This is the readme information including changes that come in with this branch. The standard documentation for this service can be found further down! + +## Compatibility Matrix + +| Keptn Version | [Dynatrace Service] | +|:----------------:|:----------------------------------------:| +| 0.6.1 | grabnerandi/dynatrace-service:0.1 | + +## Installation + +The *dynatrace-service* can either replace your existing installation or can be installed fresh in case you do not yet have a Dynatrace Service Installed. + +### New Installation if dynatrace-service not yet existing + +If you havent deployed the *dynatrace-service* yet into your Keptn installation you first need to create a secret that holds your Dynatrace Credentials. If you want to learn how to obtain tokens and tenant check out the Keptns doc: https://keptn.sh/docs/0.6.0/reference/monitoring/dynatrace/ + +Now we are ready to install this version of the dynatrace-service. We are going to use kubectl apply. Double check the version in the dynatrace-service.yaml to be the one you want to install from the list above! + +```console +kubectl apply -f deploy/manifests/dynatrace-service/dynatrace-service.yaml +``` + +When the service is deployed, use the following command to let the `dynatrace-service` install Dynatrace OneAgent on your cluster. If Dynatrace OneAgents is already deployed, the current deployment of Dynatrace will not be modified. + +```console +keptn configure monitoring dynatrace +``` + +### Replace the core dynatrace-service version with this one + +To replace the existing jmeter-service with *jmeter-extened-service* simply replace the image in the jmeter-service deployment like this +```console +kubectl -n keptn set image deployment/dynatrace-service dynatrace-service=grabnerandi/dynatrace-service:0.1 --record +``` + +If you want to revert back to the core jmeter-service do this +```console +kubectl -n keptn set image deployment/dynatrace-service dynatrace-service=keptn/dynatrace-service:0.6.2 --record +``` + +## Usage Information + +### Sending Events to Dynatrace Monitored Entities + +The *dynatrace-service* by default assumes that all events it sends to Dynatrace, e.g: Deployment or Test Start/Stop Events are sent to a monitored Dynatrace SERVICE entity that has the following attachRule definition: +``` +attachRules: + tagRule: + meTypes: + - SERVICE + tags: + - context: CONTEXTLESS + key: keptn_project + value: $PROJECT + - context: CONTEXTLESS + key: keptn_service + value: $SERVICE + - context: CONTEXTLESS + key: keptn_stage + value: $STAGE +``` + +If your services are deployed with Keptn's Helm Service chances are that your services are automatically tagged like this. Here is a screenshot of how these tags show up in Dynatrace for a service deployed with Keptn: +![](./assets/keptn_tags_in_dynatrace.png) + +If your services are however not tagged with these but other tags - or if you want the *dynatrace-service* to send the events not to a service but rather an application, process group or host then you can overwrite the default behavior by providing a *dynatrace/dynatrace.conf.yaml* file. This file can either be located on project, stage or service level. This file allows you to define your own attachRules and also allows you to leverage all available $PLACEHOLDERS such as $SERVICE,$STAGE,$PROJECT,$LABEL.YOURLABEL, ... - here is one example: It will instruct the *dynatrace-service* to send its events to a monitored Dynatrace Service that holds a tag with the key that matches your Keptn Service name ($SERICE) as well as holds an additional auto-tag that defines the enviornment to be pulled from a label that has been sent to Keptn. +``` +--- +spec_version: '0.1.0' +attachRules: + tagRule: + meTypes: + - SERVICE + tags: + - context: CONTEXTLESS + key: $SERVICE + - context: CONTEXTLESS + key: environment + value: $LABEL.environment +``` + +### Enriching Events sent to Dynatrace with more context + +The *dynatrace-service* sends CUSTOM_DEPLOYMENT, CUSTOM_INFO and CUSTOM_ANNOTATION events when it handles Keptn events such as deployment-finished, test-finished or evaluation-done. The *dynatrace-service* will parse all labels in the Keptn event and will pass them on to Dynatrace as custom properties. This gives you more flexiblity in passing more context to Dynatrace, e.g: ciBackLink for a CUSTOM_DEPLOYMENT or things like Jenkins Job ID, Jenkins Job URL ... that will show up in Dynatrace as well. +Here is a sample Deployment Finished Event: +``` +{ + "type": "sh.keptn.events.deployment-finished", + "contenttype": "application/json", + "specversion": "0.2", + "source": "jenkins", + "id": "f2b878d3-03c0-4e8f-bc3f-454bc1b3d79d", + "shkeptncontext": "08735340-6f9e-4b32-97ff-3b6c292bc509", + "data": { + "project": "simpleproject", + "stage": "staging", + "service": "simplenode", + "testStrategy": "performance", + "deploymentStrategy": "direct", + "tag": "0.10.1", + "image": "grabnerandi/simplenodeservice:1.0.0", + "labels": { + "testid": "12345", + "buildnr": "build17", + "runby": "grabnerandi", + "environment" : "testenvironment", + "ciBackLink" : "http://myjenkinsserver/job/12345" + }, + "deploymentURILocal": "http://carts.sockshop-staging.svc.cluster.local", + "deploymentURIPublic": "https://carts.sockshop-staging.my-domain.com" + } +} +``` + +It will result in the following events in Dynatrace: +![](./assets/deployevent.png) + +### Sending Events to different Dynatrace Environments per Project, Stage or Service + +Many Dynatrace user have different Dynatrace environments for e.g: Pre-Production vs Production. By default the *dynatrace-service* gets the Dynatrace Tenant URL & Token from the k8s secret stored in keptn/dynatrace (see installation instructions for details). +If you have multiple Dynatrace environment and want to have the *dynatrace-service* send events to a specific Dynatrace Environment for a specific Keptn Project, Stage or Service you can now specify the name of the secret that should be used in the *dynatrace.conf.yaml* which was introduced earlier. Here is a sample file: +``` +--- +spec_version: '0.1.0' +dtCreds: dynatrace-production +attachRules: + tagRule: + meTypes: + - SERVICE + tags: + - context: CONTEXTLESS + key: $SERVICE + - context: CONTEXTLESS + key: environment + value: $LABEL.environment +``` + +The *dtCreds* value references your k8s secret where you store your Tenant and Token information. If you do not specify dtCreds it defaults to *dynatrace* which means it is the default behavior that we had for this service since the beginning! + + + +# (STANDARD DOC) Dynatrace Service and Dynatrace OneAgent Operator ![GitHub release (latest by date)](https://img.shields.io/github/v/release/keptn-contrib/dynatrace-service) [![Build Status](https://travis-ci.org/keptn-contrib/dynatrace-service.svg?branch=master)](https://travis-ci.org/keptn-contrib/dynatrace-service) [![Go Report Card](https://goreportcard.com/badge/github.com/keptn-contrib/dynatrace-service)](https://goreportcard.com/report/github.com/keptn-contrib/dynatrace-service) @@ -69,3 +212,4 @@ To uninstall the dynatrace service and remove the subscriptions to keptn channel ```console kubectl delete -f ./deploy/manifests/dynatrace-service/dynatrace-service.yaml ``` + diff --git a/assets/deployevent.png b/assets/deployevent.png new file mode 100644 index 000000000..203d192aa Binary files /dev/null and b/assets/deployevent.png differ diff --git a/assets/keptn_tags_in_dynatrace.png b/assets/keptn_tags_in_dynatrace.png new file mode 100644 index 000000000..2aa75bd97 Binary files /dev/null and b/assets/keptn_tags_in_dynatrace.png differ diff --git a/cmd/main.go b/cmd/main.go index f381f994d..0c7cfe9ea 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -5,6 +5,7 @@ import ( "log" "os" + "github.com/keptn-contrib/dynatrace-service/pkg/common" "github.com/keptn-contrib/dynatrace-service/pkg/event_handler" "github.com/cloudevents/sdk-go/pkg/cloudevents/client" @@ -26,6 +27,11 @@ func main() { if err := envconfig.Process("", &env); err != nil { log.Fatalf("Failed to process env var: %s", err) } + + if common.RunLocal || common.RunLocalTest { + log.Println("env=runlocal: Running with local filesystem to fetch resources") + } + os.Exit(_main(os.Args[1:], env)) } @@ -38,6 +44,8 @@ func _main(args []string, env envConfig) int { cloudeventshttp.WithPath(env.Path), ) + log.Printf("Port = %d; Path=%s", env.Port, env.Path) + if err != nil { log.Fatalf("failed to create transport, %v", err) } diff --git a/deploy/manifests/dynatrace-service/dynatrace-service.yaml b/deploy/manifests/dynatrace-service/dynatrace-service.yaml index 75fcdb706..c15feb1ad 100644 --- a/deploy/manifests/dynatrace-service/dynatrace-service.yaml +++ b/deploy/manifests/dynatrace-service/dynatrace-service.yaml @@ -16,7 +16,7 @@ spec: spec: containers: - name: dynatrace-service - image: keptn/dynatrace-service:latest + image: grabnerandi/dynatrace-service:0.1 ports: - containerPort: 8080 resources: diff --git a/dynatrace/dynatrace.conf.yaml b/dynatrace/dynatrace.conf.yaml new file mode 100644 index 000000000..af94d47ca --- /dev/null +++ b/dynatrace/dynatrace.conf.yaml @@ -0,0 +1,14 @@ +--- +spec_version: '0.1.0' +dtCreds: dynatrace +attachRules: + tagRule: + - meTypes: + - SERVICE + tags: + - context: CONTEXTLESS + key: keptn_project + value: myproject + - context: CONTEXTLESS + key: keptn_service + value: myservice \ No newline at end of file diff --git a/pkg/common/common.go b/pkg/common/common.go index 8f217f2d9..7d974e099 100644 --- a/pkg/common/common.go +++ b/pkg/common/common.go @@ -1,14 +1,54 @@ package common import ( + "errors" + "os" + "k8s.io/client-go/kubernetes" "k8s.io/client-go/rest" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) +var RunLocal = (os.Getenv("env") == "runlocal") +var RunLocalTest = (os.Getenv("env") == "runlocaltest") + func GetKubernetesClient() (*kubernetes.Clientset, error) { + if RunLocal || RunLocalTest { + return nil, nil + } + config, err := rest.InClusterConfig() if err != nil { return nil, err } return kubernetes.NewForConfig(config) } + +/** + * Returns the Keptn Domain stored in the keptn-domainconfigmap + */ +func GetKeptnDomain() (string, error) { + kubeAPI, err := GetKubernetesClient() + if kubeAPI == nil || err != nil { + return "", err + } + + keptnDomainCM, errCM := kubeAPI.CoreV1().ConfigMaps("keptn").Get("keptn-domain", metav1.GetOptions{}) + if errCM != nil { + return "", errors.New("Could not retrieve keptn-domain ConfigMap: " + errCM.Error()) + } + + keptnDomain := keptnDomainCM.Data["app_domain"] + return keptnDomain, nil +} + +/** + * Returns the endpoint to the configuration-service + */ +func GetConfigurationServiceURL() string { + if os.Getenv("CONFIGURATION_SERVICE_URL") != "" { + return os.Getenv("CONFIGURATION_SERVICE_URL") + } + return "configuration-service.keptn.svc.cluster.local:8080" +} diff --git a/pkg/event_handler/cd_handler.go b/pkg/event_handler/cd_handler.go index 953edfbbd..6707443b3 100644 --- a/pkg/event_handler/cd_handler.go +++ b/pkg/event_handler/cd_handler.go @@ -15,6 +15,34 @@ type CDEventHandler struct { Event cloudevents.Event } +/** + * Initializes baseKeptnEvent and returns it + dynatraceConfig + */ +func (eh CDEventHandler) initObjectsForCDEventHandler(project, stage, service, testStrategy, image, tag string, labels map[string]string, context string) (*baseKeptnEvent, *DynatraceConfigFile, string) { + keptnEvent := &baseKeptnEvent{} + keptnEvent.project = project + keptnEvent.stage = stage + keptnEvent.service = service + keptnEvent.testStrategy = testStrategy + keptnEvent.image = image + keptnEvent.tag = tag + keptnEvent.labels = labels + keptnEvent.context = context + dynatraceConfig, _ := getDynatraceConfig(keptnEvent, eh.Logger) + keptnDomain, _ := common.GetKeptnDomain() + if keptnEvent.labels == nil { + keptnEvent.labels = make(map[string]string) + } + keptnEvent.labels["Keptns Bridge"] = "https://bridge.keptn." + keptnDomain + "/trace/" + context + + dtCreds := "" + if dynatraceConfig != nil { + dtCreds = dynatraceConfig.DtCreds + } + + return keptnEvent, dynatraceConfig, dtCreds +} + func (eh CDEventHandler) HandleEvent() error { var shkeptncontext string _ = eh.Event.Context.ExtensionAs("shkeptncontext", &shkeptncontext) @@ -47,16 +75,31 @@ func (eh CDEventHandler) HandleEvent() error { eh.Logger.Error("Could not parse event payload: " + err.Error()) return err } - de := createDeploymentEvent(dfData, shkeptncontext) - dtHelper.SendEvent(de) + // initialize our objects + keptnEvent, dynatraceConfig, dtCreds := eh.initObjectsForCDEventHandler(dfData.Project, dfData.Stage, dfData.Service, dfData.TestStrategy, dfData.Image, dfData.Tag, dfData.Labels, shkeptncontext) + if dfData.DeploymentURILocal != "" { + keptnEvent.labels["deploymentURILocal"] = dfData.DeploymentURILocal + } + if dfData.DeploymentURIPublic != "" { + keptnEvent.labels["deploymentURIPublic"] = dfData.DeploymentURIPublic + } + + // send Deployment EVent + de := createDeploymentEvent(keptnEvent, dynatraceConfig, eh.Logger) + dtHelper.SendEvent(de, dtCreds) // TODO: an additional channel (e.g. start-tests) to correctly determine the time when the tests actually start - ie := createInfoEvent(dfData.Project, dfData.Stage, dfData.Service, dfData.TestStrategy, dfData.Image, dfData.Tag, shkeptncontext) + // ie := createInfoEvent(keptnEvent, eh.Logger) + ie := createAnnotationEvent(keptnEvent, dynatraceConfig, eh.Logger) if dfData.TestStrategy != "" { - ie.Title = "Start Running Tests: " + dfData.TestStrategy - ie.Description = "Start running tests: " + dfData.TestStrategy + " against " + dfData.Service - dtHelper.SendEvent(ie) + if ie.AnnotationType == "" { + ie.AnnotationType = "Start Tests: " + dfData.TestStrategy + } + if ie.AnnotationDescription == "" { + ie.AnnotationDescription = "Start running tests: " + dfData.TestStrategy + " against " + dfData.Service + } + dtHelper.SendEvent(ie, dtCreds) } } else if eh.Event.Type() == keptn.TestsFinishedEventType { tfData := &keptn.TestsFinishedEventData{} @@ -65,11 +108,22 @@ func (eh CDEventHandler) HandleEvent() error { eh.Logger.Error("Could not parse event payload: " + err.Error()) return err } - ie := createInfoEvent(tfData.Project, tfData.Stage, tfData.Service, tfData.TestStrategy, "", "", shkeptncontext) - ie.Title = "Stop Running Tests: " + tfData.TestStrategy - ie.Description = "Stop running tests: " + tfData.TestStrategy + " against " + tfData.Service - dtHelper.SendEvent(ie) + // initialize our objects + keptnEvent, dynatraceConfig, dtCreds := eh.initObjectsForCDEventHandler(tfData.Project, tfData.Stage, tfData.Service, tfData.TestStrategy, "", "", tfData.Labels, shkeptncontext) + + // Send Annotation Event + // ie := createInfoEvent(keptnEvent, eh.Logger) + ie := createAnnotationEvent(keptnEvent, dynatraceConfig, eh.Logger) + if tfData.TestStrategy != "" { + if ie.AnnotationType == "" { + ie.AnnotationType = "Stop Tests: " + tfData.TestStrategy + } + if ie.AnnotationDescription == "" { + ie.AnnotationDescription = "Stop running tests: " + tfData.TestStrategy + " against " + tfData.Service + } + dtHelper.SendEvent(ie, dtCreds) + } } else if eh.Event.Type() == keptn.EvaluationDoneEventType { edData := &keptn.EvaluationDoneEventData{} err := eh.Event.DataAs(edData) @@ -77,12 +131,25 @@ func (eh CDEventHandler) HandleEvent() error { fmt.Println("Error while parsing JSON payload: " + err.Error()) return err } - ie := createInfoEvent(edData.Project, edData.Stage, edData.Service, edData.TestStrategy, "", "", shkeptncontext) - if edData.Result == "pass" || edData.Result == "warning" { + + // initialize our objects + keptnEvent, dynatraceConfig, dtCreds := eh.initObjectsForCDEventHandler(edData.Project, edData.Stage, edData.Service, edData.TestStrategy, "", "", edData.Labels, shkeptncontext) + keptnEvent.labels["Quality Gate Score"] = fmt.Sprintf("%.2f", edData.EvaluationDetails.Score) + keptnEvent.labels["No of evaluated SLIs"] = fmt.Sprintf("%d", len(edData.EvaluationDetails.IndicatorResults)) + keptnEvent.labels["Evaluation Start"] = edData.EvaluationDetails.TimeStart + keptnEvent.labels["Evaluation End"] = edData.EvaluationDetails.TimeEnd + + // Send Info Event + ie := createInfoEvent(keptnEvent, dynatraceConfig, eh.Logger) + // If DeploymentStrategy == "" it means we are doing Quality-Gates Only! + if edData.DeploymentStrategy == "" { + ie.Title = fmt.Sprintf("Quality Gate Result: %s (%.2f/100)", edData.Result, edData.EvaluationDetails.Score) + } else if edData.Result == "pass" || edData.Result == "warning" { if edData.TestStrategy == "real-user" { ie.Title = "Remediation action successful" } else { - ie.Title = "Promote Artifact from " + edData.Stage + " to next stage" + ie.Title = fmt.Sprintf("Quality Gate Result: %s (%.2f/100): PROMOTING from %s to next stage", edData.Result, edData.EvaluationDetails.Score, edData.Stage) + // ie.Title = "Promote Artifact from " + edData.Stage + " to next stage" } } else if edData.Result == "fail" && edData.DeploymentStrategy == "blue_green_service" { @@ -95,14 +162,15 @@ func (eh CDEventHandler) HandleEvent() error { if edData.TestStrategy == "real-user" { ie.Title = "Remediation action not successful" } else { - ie.Title = "NOT PROMOTING Artifact from " + edData.Stage + " due to failed evaluation" + ie.Title = fmt.Sprintf("Quality Gate Result: %s (%.2f/100): NOT PROMOTING artifact from %s", edData.Result, edData.EvaluationDetails.Score, edData.Stage) + // ie.Title = "NOT PROMOTING Artifact from " + edData.Stage + " due to failed evaluation" } } else { eh.Logger.Error("No valid deployment strategy defined in keptn event.") return nil } ie.Description = "Keptn evaluation status: " + edData.Result - dtHelper.SendEvent(ie) + dtHelper.SendEvent(ie, dtCreds) } else { eh.Logger.Info(" Ignoring event.") } @@ -110,18 +178,18 @@ func (eh CDEventHandler) HandleEvent() error { } type dtTag struct { - Context string `json:"context"` - Key string `json:"key"` - Value string `json:"value"` + Context string `json:"context" yaml:"context"` + Key string `json:"key" yaml:"key"` + Value string `json:"value",omitempty yaml:"value",omitempty` } type dtTagRule struct { - MeTypes []string `json:"meTypes"` - Tags []dtTag `json:"tags"` + MeTypes []string `json:"meTypes" yaml:"meTypes"` + Tags []dtTag `json:"tags" yaml:"tags"` } type dtAttachRules struct { - TagRule []dtTagRule `json:"tagRule"` + TagRule []dtTagRule `json:"tagRule" yaml:"tagRule"` } type dtCustomProperties struct { @@ -136,25 +204,46 @@ type dtCustomProperties struct { } type dtDeploymentEvent struct { - EventType string `json:"eventType"` - Source string `json:"source"` - AttachRules dtAttachRules `json:"attachRules"` - CustomProperties dtCustomProperties `json:"customProperties"` - DeploymentVersion string `json:"deploymentVersion"` - DeploymentName string `json:"deploymentName"` - DeploymentProject string `json:"deploymentProject"` + EventType string `json:"eventType"` + Source string `json:"source"` + AttachRules dtAttachRules `json:"attachRules"` + // CustomProperties dtCustomProperties `json:"customProperties"` + CustomProperties map[string]string `json:"customProperties"` + DeploymentVersion string `json:"deploymentVersion"` + DeploymentName string `json:"deploymentName"` + DeploymentProject string `json:"deploymentProject"` + CiBackLink string `json:"ciBackLink",omitempty` + RemediationAction string `json:"remediationAction",omitempty` } type dtInfoEvent struct { - EventType string `json:"eventType"` - Source string `json:"source"` - AttachRules dtAttachRules `json:"attachRules"` - CustomProperties dtCustomProperties `json:"customProperties"` - Description string `json:"description"` - Title string `json:"title"` + EventType string `json:"eventType"` + Source string `json:"source"` + AttachRules dtAttachRules `json:"attachRules"` + // CustomProperties dtCustomProperties `json:"customProperties"` + CustomProperties map[string]string `json:"customProperties"` + Description string `json:"description"` + Title string `json:"title"` +} + +type dtAnnotationEvent struct { + EventType string `json:"eventType"` + Source string `json:"source"` + AttachRules dtAttachRules `json:"attachRules"` + // CustomProperties dtCustomProperties `json:"customProperties"` + CustomProperties map[string]string `json:"customProperties"` + AnnotationDescription string `json:"annotationDescription"` + AnnotationType string `json:"annotationType"` } -func createAttachRules(project string, stage string, service string) dtAttachRules { +/** + * Changes in #115_116: Parse Tags from dynatrace.conf.yaml and only fall back to default behavior if it doesnt exist + */ +func createAttachRules(keptnEvent *baseKeptnEvent, dynatraceConfig *DynatraceConfigFile, logger *keptn.Logger) dtAttachRules { + if dynatraceConfig != nil && dynatraceConfig.AttachRules != nil { + return *dynatraceConfig.AttachRules + } + ar := dtAttachRules{ TagRule: []dtTagRule{ dtTagRule{ @@ -163,61 +252,138 @@ func createAttachRules(project string, stage string, service string) dtAttachRul dtTag{ Context: "CONTEXTLESS", Key: "keptn_project", - Value: project, + Value: keptnEvent.project, }, dtTag{ Context: "CONTEXTLESS", Key: "keptn_stage", - Value: stage, + Value: keptnEvent.stage, }, dtTag{ Context: "CONTEXTLESS", Key: "keptn_service", - Value: service, + Value: keptnEvent.service, }, }, }, }, } + return ar } -func createCustomProperties(project string, stage string, service string, testStrategy string, image string, tag string, keptnContext string) dtCustomProperties { - var customProperties dtCustomProperties - customProperties.Project = project - customProperties.Stage = stage - customProperties.Service = service - customProperties.TestStrategy = testStrategy - customProperties.Image = image - customProperties.Tag = tag - customProperties.KeptnContext = keptnContext +/** + * Change with #115_116: parse labels and move them into custom properties + */ +// func createCustomProperties(project string, stage string, service string, testStrategy string, image string, tag string, labels map[string]string, keptnContext string) dtCustomProperties { +func createCustomProperties(keptnEvent *baseKeptnEvent, logger *keptn.Logger) map[string]string { + // TODO: AG - parse labels and push them through + + // var customProperties dtCustomProperties + // customProperties.Project = project + // customProperties.Stage = stage + // customProperties.Service = service + // customProperties.TestStrategy = testStrategy + // customProperties.Image = image + // customProperties.Tag = tag + // customProperties.KeptnContext = keptnContext + var customProperties map[string]string + customProperties = make(map[string]string) + customProperties["Project"] = keptnEvent.project + customProperties["Stage"] = keptnEvent.stage + customProperties["Service"] = keptnEvent.service + customProperties["TestStrategy"] = keptnEvent.testStrategy + customProperties["Image"] = keptnEvent.image + customProperties["Tag"] = keptnEvent.tag + customProperties["KeptnContext"] = keptnEvent.context + + // now add the rest of the labels + for key, value := range keptnEvent.labels { + customProperties[key] = value + } + return customProperties } -func createInfoEvent(project string, stage string, service string, testStrategy string, image string, tag string, keptnContext string) dtInfoEvent { - ar := createAttachRules(project, stage, service) - customProperties := createCustomProperties(project, stage, service, testStrategy, image, tag, keptnContext) +/** + * Returns the value of the map if the value exists - otherwise returns default + * Also removes the found value from the map if removeIfFound==true + */ +func getValueFromLabels(labels *map[string]string, valueKey string, defaultValue string, removeIfFound bool) string { + mapValue, mapValueOk := (*labels)[valueKey] + if mapValueOk { + if removeIfFound { + delete(*labels, valueKey) + } + return mapValue + } + + return defaultValue +} + +// project string, stage string, service string, testStrategy string, image string, tag string, labels map[string]string, keptnContext string +func createInfoEvent(keptnEvent *baseKeptnEvent, dynatraceConfig *DynatraceConfigFile, logger *keptn.Logger) dtInfoEvent { + // we fill the Dynatrace Info Event with values from the labels or use our defaults var ie dtInfoEvent + ie.EventType = "CUSTOM_INFO" + ie.Source = "Keptn dynatrace-service" + ie.Title = getValueFromLabels(&keptnEvent.labels, "title", "", true) + ie.Description = getValueFromLabels(&keptnEvent.labels, "description", "", true) + + // now we create our attach rules + ar := createAttachRules(keptnEvent, dynatraceConfig, logger) ie.AttachRules = ar + + // and add the rest of the labels and info as custom properties + customProperties := createCustomProperties(keptnEvent, logger) ie.CustomProperties = customProperties - ie.EventType = "CUSTOM_INFO" + + return ie +} + +/** + * Creates a Dynatrace ANNOTATION event + */ +func createAnnotationEvent(keptnEvent *baseKeptnEvent, dynatraceConfig *DynatraceConfigFile, logger *keptn.Logger) dtAnnotationEvent { + + // we fill the Dynatrace Info Event with values from the labels or use our defaults + var ie dtAnnotationEvent + ie.EventType = "CUSTOM_ANNOTATION" ie.Source = "Keptn dynatrace-service" + ie.AnnotationType = getValueFromLabels(&keptnEvent.labels, "type", "", true) + ie.AnnotationDescription = getValueFromLabels(&keptnEvent.labels, "description", "", true) + + // now we create our attach rules + ar := createAttachRules(keptnEvent, dynatraceConfig, logger) + ie.AttachRules = ar + + // and add the rest of the labels and info as custom properties + customProperties := createCustomProperties(keptnEvent, logger) + ie.CustomProperties = customProperties return ie } -func createDeploymentEvent(event *keptn.DeploymentFinishedEventData, keptnContext string) dtDeploymentEvent { - ar := createAttachRules(event.Project, event.Stage, event.Service) - customProperties := createCustomProperties(event.Project, event.Stage, event.Service, event.TestStrategy, event.Image, event.Tag, keptnContext) +func createDeploymentEvent(keptnEvent *baseKeptnEvent, dynatraceConfig *DynatraceConfigFile, logger *keptn.Logger) dtDeploymentEvent { + // we fill the Dynatrace Deployment Event with values from the labels or use our defaults var de dtDeploymentEvent de.EventType = "CUSTOM_DEPLOYMENT" de.Source = "Keptn dynatrace-service" - de.DeploymentName = "Deploy " + event.Service + " " + event.Tag + " with strategy " + event.DeploymentStrategy - de.DeploymentProject = event.Project - de.DeploymentVersion = event.Tag + de.DeploymentName = getValueFromLabels(&keptnEvent.labels, "deploymentName", "Deploy "+keptnEvent.service+" "+keptnEvent.tag+" with strategy "+keptnEvent.deploymentStrategy, true) + de.DeploymentProject = getValueFromLabels(&keptnEvent.labels, "deploymentProject", keptnEvent.project, true) + de.DeploymentVersion = getValueFromLabels(&keptnEvent.labels, "deploymentVersion", keptnEvent.tag, true) + de.CiBackLink = getValueFromLabels(&keptnEvent.labels, "ciBackLink", "", true) + de.RemediationAction = getValueFromLabels(&keptnEvent.labels, "remediationAction", "", true) + + // now we create our attach rules + ar := createAttachRules(keptnEvent, dynatraceConfig, logger) de.AttachRules = ar + + // and add the rest of the labels and info as custom properties + // TODO: event.Project, event.Stage, event.Service, event.TestStrategy, event.Image, event.Tag, event.Labels, keptnContext + customProperties := createCustomProperties(keptnEvent, logger) de.CustomProperties = customProperties return de diff --git a/pkg/event_handler/handler.go b/pkg/event_handler/handler.go index 92a98f0b3..10fafac4a 100644 --- a/pkg/event_handler/handler.go +++ b/pkg/event_handler/handler.go @@ -1,10 +1,51 @@ package event_handler import ( + "errors" + "fmt" + "io/ioutil" + "os" + "strings" + + "github.com/keptn-contrib/dynatrace-service/pkg/common" + "github.com/cloudevents/sdk-go/pkg/cloudevents" + "github.com/ghodss/yaml" keptn "github.com/keptn/go-utils/pkg/lib" + + utils "github.com/keptn/go-utils/pkg/api/utils" ) +const DynatraceConfigFilename = "dynatrace/dynatrace.conf.yaml" +const DynatraceConfigFilenameLOCAL = "dynatrace/_dynatrace.conf.yaml" + +/** + * Defines the Dynatrace Configuration File structure! + */ +type DynatraceConfigFile struct { + SpecVersion string `json:"spec_version" yaml:"spec_version"` + DtCreds string `json:"dtCreds",omitempty yaml:"dtCreds",omitempty` + AttachRules *dtAttachRules `json:"attachRules",omitempty yaml:"attachRules",omitempty` +} + +type baseKeptnEvent struct { + context string + source string + event string + + project string + stage string + service string + deployment string + testStrategy string + deploymentStrategy string + + image string + tag string + + labels map[string]string +} + type DynatraceEventHandler interface { HandleEvent() error } @@ -22,3 +63,115 @@ func NewEventHandler(event cloudevents.Event, logger *keptn.Logger) (DynatraceEv return &CDEventHandler{Logger: logger, Event: event}, nil } } + +// +// replaces $ placeholders with actual values +// $CONTEXT, $EVENT, $SOURCE +// $PROJECT, $STAGE, $SERVICE, $DEPLOYMENT +// $TESTSTRATEGY +// $LABEL.XXXX -> will replace that with a label called XXXX +// $ENV.XXXX -> will replace that with an env variable called XXXX +// $SECRET.YYYY -> will replace that with the k8s secret called YYYY +// +func replaceKeptnPlaceholders(input string, keptnEvent *baseKeptnEvent) string { + result := input + + // first we do the regular keptn values + result = strings.Replace(result, "$CONTEXT", keptnEvent.context, -1) + result = strings.Replace(result, "$EVENT", keptnEvent.event, -1) + result = strings.Replace(result, "$SOURCE", keptnEvent.source, -1) + result = strings.Replace(result, "$PROJECT", keptnEvent.project, -1) + result = strings.Replace(result, "$STAGE", keptnEvent.stage, -1) + result = strings.Replace(result, "$SERVICE", keptnEvent.service, -1) + result = strings.Replace(result, "$DEPLOYMENT", keptnEvent.deployment, -1) + result = strings.Replace(result, "$TESTSTRATEGY", keptnEvent.testStrategy, -1) + + // now we do the labels + for key, value := range keptnEvent.labels { + result = strings.Replace(result, "$LABEL."+key, value, -1) + } + + // now we do all environment variables + for _, env := range os.Environ() { + pair := strings.SplitN(env, "=", 2) + result = strings.Replace(result, "$ENV."+pair[0], pair[1], -1) + } + + // TODO: iterate through k8s secrets! + + return result +} + +// +// Loads dynatrace.conf for the current service +// +func getDynatraceConfig(keptnEvent *baseKeptnEvent, logger *keptn.Logger) (*DynatraceConfigFile, error) { + + logger.Info("Loading dynatrace.conf.yaml") + // if we run in a runlocal mode we are just getting the file from the local disk + var fileContent string + if common.RunLocal { + localFileContent, err := ioutil.ReadFile(DynatraceConfigFilenameLOCAL) + if err != nil { + logMessage := fmt.Sprintf("No %s file found LOCALLY for service %s in stage %s in project %s", DynatraceConfigFilenameLOCAL, keptnEvent.service, keptnEvent.stage, keptnEvent.project) + logger.Info(logMessage) + return nil, nil + } + logger.Info("Loaded LOCAL file " + DynatraceConfigFilenameLOCAL) + fileContent = string(localFileContent) + } else { + resourceHandler := utils.NewResourceHandler(common.GetConfigurationServiceURL()) + + // Lets search on SERVICE-LEVEL + keptnResourceContent, err := resourceHandler.GetServiceResource(keptnEvent.project, keptnEvent.stage, keptnEvent.service, DynatraceConfigFilename) + if err != nil || keptnResourceContent == nil || keptnResourceContent.ResourceContent == "" { + // Lets search on STAGE-LEVEL + keptnResourceContent, err = resourceHandler.GetStageResource(keptnEvent.project, keptnEvent.stage, DynatraceConfigFilename) + if err != nil || keptnResourceContent == nil || keptnResourceContent.ResourceContent == "" { + // Lets search on PROJECT-LEVEL + keptnResourceContent, err = resourceHandler.GetProjectResource(keptnEvent.project, DynatraceConfigFilename) + if err != nil || keptnResourceContent == nil || keptnResourceContent.ResourceContent == "" { + logger.Debug(fmt.Sprintf("No Keptn Resource found: %s/%s/%s/%s - %s", keptnEvent.project, keptnEvent.stage, keptnEvent.service, DynatraceConfigFilename, err)) + return nil, err + } + + logger.Debug("Found " + DynatraceConfigFilename + " on project level") + } else { + logger.Debug("Found " + DynatraceConfigFilename + " on stage level") + } + } else { + logger.Debug("Found " + DynatraceConfigFilename + " on service level") + } + fileContent = keptnResourceContent.ResourceContent + } + + // replace the placeholders + logger.Debug("Content of dynatrace.conf.yaml: " + fileContent) + fileContent = replaceKeptnPlaceholders(fileContent, keptnEvent) + logger.Debug("After replacements: " + fileContent) + + // unmarshal the file + dynatraceConfFile, err := parseDynatraceConfigFile([]byte(fileContent)) + + if err != nil { + logMessage := fmt.Sprintf("Couldn't parse %s file found for service %s in stage %s in project %s. Error: %s", DynatraceConfigFilename, keptnEvent.service, keptnEvent.stage, keptnEvent.project, err.Error()) + logger.Error(logMessage) + return nil, errors.New(logMessage) + } + + logMessage := fmt.Sprintf("Loaded Config from dynatrace.conf.yaml: %s", dynatraceConfFile) + logger.Info(logMessage) + + return dynatraceConfFile, nil +} + +func parseDynatraceConfigFile(input []byte) (*DynatraceConfigFile, error) { + dynatraceConfFile := &DynatraceConfigFile{} + err := yaml.Unmarshal([]byte(input), &dynatraceConfFile) + + if err != nil { + return nil, err + } + + return dynatraceConfFile, nil +} diff --git a/pkg/lib/auto_tags.go b/pkg/lib/auto_tags.go index 2278dc01c..09636f39f 100644 --- a/pkg/lib/auto_tags.go +++ b/pkg/lib/auto_tags.go @@ -5,7 +5,7 @@ import "encoding/json" func (dt *DynatraceHelper) EnsureDTTaggingRulesAreSetUp() error { dt.Logger.Info("Setting up auto-tagging rules in Dynatrace Tenant") - response, err := dt.sendDynatraceAPIRequest("/api/config/v1/autoTags", "GET", "") + response, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/autoTags", "GET", "") existingDTRules := &DTAPIListResponse{} @@ -34,7 +34,7 @@ func (dt *DynatraceHelper) createDTTaggingRule(rule *DTTaggingRule) error { if err != nil { return err } - _, err = dt.sendDynatraceAPIRequest("/api/config/v1/autoTags", "POST", string(payload)) + _, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/autoTags", "POST", string(payload)) return err } @@ -42,7 +42,7 @@ func (dt *DynatraceHelper) deleteExistingDTTaggingRule(ruleName string, existing dt.Logger.Info("Deleting rule " + ruleName) for _, rule := range existingRules.Values { if rule.Name == ruleName { - _, err := dt.sendDynatraceAPIRequest("/api/config/v1/autoTags/"+rule.ID, "DELETE", "") + _, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/autoTags/"+rule.ID, "DELETE", "") if err != nil { dt.Logger.Info("Could not delete rule " + rule.ID + ": " + err.Error()) } diff --git a/pkg/lib/dashboard.go b/pkg/lib/dashboard.go index a6f44d068..19fd7a849 100644 --- a/pkg/lib/dashboard.go +++ b/pkg/lib/dashboard.go @@ -3,20 +3,15 @@ package lib import ( "encoding/json" + "github.com/keptn-contrib/dynatrace-service/pkg/common" keptn "github.com/keptn/go-utils/pkg/lib" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) func (dt *DynatraceHelper) CreateDashboard(project string, shipyard keptn.Shipyard, services []string) error { - keptnDomainCM, err := dt.KubeApi.CoreV1().ConfigMaps("keptn").Get("keptn-domain", metav1.GetOptions{}) - if err != nil { - dt.Logger.Error("Could not retrieve keptn-domain ConfigMap: " + err.Error()) - } - - keptnDomain := keptnDomainCM.Data["app_domain"] + keptnDomain, _ := common.GetKeptnDomain() // first, check if dashboard for this project already exists and delete that - err = dt.DeleteExistingDashboard(project) + err := dt.DeleteExistingDashboard(project) if err != nil { return err } @@ -29,7 +24,7 @@ func (dt *DynatraceHelper) CreateDashboard(project string, shipyard keptn.Shipya dashboardPayload, _ := json.Marshal(dashboard) - _, err = dt.sendDynatraceAPIRequest("/api/config/v1/dashboards", "POST", string(dashboardPayload)) + _, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/dashboards", "POST", string(dashboardPayload)) if err != nil { return err @@ -39,7 +34,7 @@ func (dt *DynatraceHelper) CreateDashboard(project string, shipyard keptn.Shipya } func (dt *DynatraceHelper) DeleteExistingDashboard(project string) error { - res, err := dt.sendDynatraceAPIRequest("/api/config/v1/dashboards", "GET", "") + res, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/dashboards", "GET", "") if err != nil { dt.Logger.Error("Could not retrieve list of existing Dynatrace dashboards: " + err.Error()) return err @@ -55,7 +50,7 @@ func (dt *DynatraceHelper) DeleteExistingDashboard(project string) error { for _, dashboardItem := range dtDashboardsResponse.Dashboards { if dashboardItem.Name == project+"@keptn: Digital Delivery & Operations Dashboard" { - res, err = dt.sendDynatraceAPIRequest("/api/config/v1/dashboards/"+dashboardItem.ID, "DELETE", "") + res, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/dashboards/"+dashboardItem.ID, "DELETE", "") if err != nil { dt.Logger.Error("Could not delete previous dashboard for project " + project + ": " + err.Error()) return err diff --git a/pkg/lib/dynatrace.go b/pkg/lib/dynatrace.go index c5d63f950..ccabee15d 100644 --- a/pkg/lib/dynatrace.go +++ b/pkg/lib/dynatrace.go @@ -52,11 +52,12 @@ type DynatraceHelper struct { Logger keptn.LoggerInterface OperatorTag string KeptnHandler *keptn.Keptn + KeptnBridge string } func NewDynatraceHelper(keptnHandler *keptn.Keptn) (*DynatraceHelper, error) { dtHelper := &DynatraceHelper{} - dtCreds, err := dtHelper.GetDTCredentials() + dtCreds, err := dtHelper.GetDTCredentials("") if err != nil { return nil, err } @@ -69,7 +70,7 @@ func (dt *DynatraceHelper) CreateCalculatedMetrics(project string) error { dt.Logger.Info("creating metric calc:service.topurlresponsetime" + project) responseTimeMetric := CreateCalculatedMetric("calc:service.topurlresponsetime"+project, "Top URL Response Time", "RESPONSE_TIME", "MICRO_SECOND", "CONTEXTLESS", "keptn_project", project, "URL", "{URL:Path}", "SUM") responseTimeJSONPayload, _ := json.Marshal(&responseTimeMetric) - _, err := dt.sendDynatraceAPIRequest("/api/config/v1/customMetric/service/"+"calc:service.topurlresponsetime"+project, "PUT", string(responseTimeJSONPayload)) + _, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/customMetric/service/"+"calc:service.topurlresponsetime"+project, "PUT", string(responseTimeJSONPayload)) if err != nil { dt.Logger.Error("could not create calculated metric calc:service.topurlresponsetime" + project + ". " + err.Error()) } @@ -77,7 +78,7 @@ func (dt *DynatraceHelper) CreateCalculatedMetrics(project string) error { dt.Logger.Info("creating metric calc:service.topurlservicecalls" + project) topServiceCalls := CreateCalculatedMetric("calc:service.topurlservicecalls"+project, "Top URL Service Calls", "NON_DATABASE_CHILD_CALL_COUNT", "COUNT", "CONTEXTLESS", "keptn_project", project, "URL", "{URL:Path}", "SINGLE_VALUE") topServiceCallsJSONPayload, _ := json.Marshal(&topServiceCalls) - _, err = dt.sendDynatraceAPIRequest("/api/config/v1/customMetric/service/"+"calc:service.topurlservicecalls"+project, "PUT", string(topServiceCallsJSONPayload)) + _, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/customMetric/service/"+"calc:service.topurlservicecalls"+project, "PUT", string(topServiceCallsJSONPayload)) if err != nil { dt.Logger.Error("could not create calculated metric calc:service.topurlservicecalls" + project + ". " + err.Error()) } @@ -85,7 +86,7 @@ func (dt *DynatraceHelper) CreateCalculatedMetrics(project string) error { dt.Logger.Info("creating metric calc:service.topurldbcalls" + project) topDBCalls := CreateCalculatedMetric("calc:service.topurldbcalls"+project, "Top URL DB Calls", "DATABASE_CHILD_CALL_COUNT", "COUNT", "CONTEXTLESS", "keptn_project", project, "URL", "{URL:Path}", "SINGLE_VALUE") topDBCallsJSONPayload, _ := json.Marshal(&topDBCalls) - _, err = dt.sendDynatraceAPIRequest("/api/config/v1/customMetric/service/"+"calc:service.topurldbcalls"+project, "PUT", string(topDBCallsJSONPayload)) + _, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/customMetric/service/"+"calc:service.topurldbcalls"+project, "PUT", string(topDBCallsJSONPayload)) if err != nil { dt.Logger.Error("could not create calculated metric calc:service.topurldbcalls" + project + ". " + err.Error()) } @@ -97,7 +98,7 @@ func (dt *DynatraceHelper) CreateTestStepCalculatedMetrics(project string) error dt.Logger.Info("creating metric calc:service.teststepresponsetime" + project) responseTimeMetric := CreateCalculatedTestStepMetric("calc:service.teststepresponsetime"+project, "Test Step Response Time", "RESPONSE_TIME", "MICRO_SECOND", "CONTEXTLESS", "keptn_project", project, "URL", "{URL:Path}", "SUM") responseTimeJSONPayload, _ := json.Marshal(&responseTimeMetric) - _, err := dt.sendDynatraceAPIRequest("/api/config/v1/customMetric/service/"+"calc:service.teststepresponsetime"+project, "PUT", string(responseTimeJSONPayload)) + _, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/customMetric/service/"+"calc:service.teststepresponsetime"+project, "PUT", string(responseTimeJSONPayload)) if err != nil { dt.Logger.Error("could not create calculated metric calc:service.teststepresponsetime" + project + ". " + err.Error()) } @@ -105,7 +106,7 @@ func (dt *DynatraceHelper) CreateTestStepCalculatedMetrics(project string) error dt.Logger.Info("creating metric calc:service.teststepservicecalls" + project) topServiceCalls := CreateCalculatedTestStepMetric("calc:service.teststepservicecalls"+project, "Test Step Service Calls", "NON_DATABASE_CHILD_CALL_COUNT", "COUNT", "CONTEXTLESS", "keptn_project", project, "URL", "{URL:Path}", "SINGLE_VALUE") topServiceCallsJSONPayload, _ := json.Marshal(&topServiceCalls) - _, err = dt.sendDynatraceAPIRequest("/api/config/v1/customMetric/service/"+"calc:service.teststepservicecalls"+project, "PUT", string(topServiceCallsJSONPayload)) + _, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/customMetric/service/"+"calc:service.teststepservicecalls"+project, "PUT", string(topServiceCallsJSONPayload)) if err != nil { dt.Logger.Error("could not create calculated metric calc:service.teststepservicecalls" + project + ". " + err.Error()) } @@ -113,7 +114,7 @@ func (dt *DynatraceHelper) CreateTestStepCalculatedMetrics(project string) error dt.Logger.Info("creating metric calc:service.teststepdbcalls" + project) topDBCalls := CreateCalculatedTestStepMetric("calc:service.teststepdbcalls"+project, "Test Step DB Calls", "DATABASE_CHILD_CALL_COUNT", "COUNT", "CONTEXTLESS", "keptn_project", project, "URL", "{URL:Path}", "SINGLE_VALUE") topDBCallsJSONPayload, _ := json.Marshal(&topDBCalls) - _, err = dt.sendDynatraceAPIRequest("/api/config/v1/customMetric/service/"+"calc:service.teststepdbcalls"+project, "PUT", string(topDBCallsJSONPayload)) + _, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/customMetric/service/"+"calc:service.teststepdbcalls"+project, "PUT", string(topDBCallsJSONPayload)) if err != nil { dt.Logger.Error("could not create calculated metric calc:service.teststepdbcalls" + project + ". " + err.Error()) } @@ -121,7 +122,7 @@ func (dt *DynatraceHelper) CreateTestStepCalculatedMetrics(project string) error dt.Logger.Info("creating metric calc:service.teststepfailurerate" + project) failureRate := CreateCalculatedTestStepMetric("calc:service.teststepfailurerate"+project, "Test Step DB Calls", "FAILURE_RATE", "PERCENT", "CONTEXTLESS", "keptn_project", project, "URL", "{URL:Path}", "OF_INTEREST_RATIO") failureRateJSONPayload, _ := json.Marshal(&failureRate) - _, err = dt.sendDynatraceAPIRequest("/api/config/v1/customMetric/service/"+"calc:service.teststepfailurerate"+project, "PUT", string(failureRateJSONPayload)) + _, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/customMetric/service/"+"calc:service.teststepfailurerate"+project, "PUT", string(failureRateJSONPayload)) if err != nil { dt.Logger.Error("could not create calculated metric calc:service.teststepfailurerate" + project + ". " + err.Error()) } @@ -143,7 +144,7 @@ func (dt *DynatraceHelper) CreateManagementZones(project string, shipyard keptn. if !found { managementZone := CreateManagementZoneForProject(project) mzPayload, _ := json.Marshal(managementZone) - _, err := dt.sendDynatraceAPIRequest("/api/config/v1/managementZones", "POST", string(mzPayload)) + _, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/managementZones", "POST", string(mzPayload)) if err != nil { dt.Logger.Error("Could not create management zone: " + err.Error()) } @@ -160,7 +161,7 @@ func (dt *DynatraceHelper) CreateManagementZones(project string, shipyard keptn. if !found { managementZone := CreateManagementZoneForStage(project, stage.Name) mzPayload, _ := json.Marshal(managementZone) - _, err := dt.sendDynatraceAPIRequest("/api/config/v1/managementZones", "POST", string(mzPayload)) + _, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/managementZones", "POST", string(mzPayload)) if err != nil { dt.Logger.Error("Could not create management zone: " + err.Error()) } @@ -175,7 +176,7 @@ func getManagementZoneNameForStage(project string, stage string) string { } func (dt *DynatraceHelper) getManagementZones() *DTAPIListResponse { - response, err := dt.sendDynatraceAPIRequest("/api/config/v1/managementZones", "GET", "") + response, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/managementZones", "GET", "") if err != nil { dt.Logger.Error("Could not retrieve management zones: " + err.Error()) } @@ -188,11 +189,30 @@ func (dt *DynatraceHelper) getManagementZones() *DTAPIListResponse { return mzs } -func (dt *DynatraceHelper) sendDynatraceAPIRequest(apiPath string, method string, body string) (string, error) { - req, err := http.NewRequest(method, "https://"+dt.DynatraceCreds.Tenant+apiPath, strings.NewReader(body)) +/** + * if dtCredsSecretName is passed and it is not dynatrace (=default) then we try to pull the secret based on that name and is it for this API Call + */ +func (dt *DynatraceHelper) sendDynatraceAPIRequest(dtCredsSecretName string, apiPath string, method string, body string) (string, error) { + + // Check if we have to use a different dynatrace credential for this call other than default + dtCredentials := dt.DynatraceCreds + if dtCredsSecretName != "dynatrace" && dtCredsSecretName != "" { + var err error + dtCredentials, err = dt.GetDTCredentials(dtCredsSecretName) + if err != nil { + dt.Logger.Error("couldnt retrieve Dynatrace Credentials from custom secret " + dtCredsSecretName + ": " + err.Error()) + } + } + + if common.RunLocal || common.RunLocalTest { + dt.Logger.Info("Dynatrace.sendDynatraceAPIRequest(RUNLOCAL) - not sending event to " + dtCredsSecretName + "(" + dtCredentials.Tenant + "). Here is the payload: " + body) + return "", nil + } + + req, err := http.NewRequest(method, "https://"+dtCredentials.Tenant+apiPath, strings.NewReader(body)) req.Header.Set("Content-Type", "application/json") - req.Header.Set("Authorization", "Api-Token "+dt.DynatraceCreds.ApiToken) + req.Header.Set("Authorization", "Api-Token "+dtCredentials.ApiToken) req.Header.Set("User-Agent", "keptn-contrib/dynatrace-service:"+os.Getenv("version")) client := &http.Client{} @@ -215,12 +235,28 @@ func (dt *DynatraceHelper) sendDynatraceAPIRequest(apiPath string, method string return string(responseBody), nil } -func (dt *DynatraceHelper) GetDTCredentials() (*DTCredentials, error) { +/** + * Pulls the Dynatrace Credentials from the passed secret. The default is "dynatrace" + */ +func (dt *DynatraceHelper) GetDTCredentials(dynatraceSecretName string) (*DTCredentials, error) { + if dynatraceSecretName == "" { + dynatraceSecretName = "dynatrace" + } + + if common.RunLocal || common.RunLocalTest { + dtCreds := &DTCredentials{} + + dtCreds.Tenant = os.Getenv("DT_TENANT") + dtCreds.ApiToken = os.Getenv("DT_API_TOKEN") + dtCreds.PaaSToken = os.Getenv("DT_PAAS_TOKEN") + return dtCreds, nil + } + kubeAPI, err := common.GetKubernetesClient() if err != nil { return nil, err } - secret, err := kubeAPI.CoreV1().Secrets("keptn").Get("dynatrace", metav1.GetOptions{}) + secret, err := kubeAPI.CoreV1().Secrets("keptn").Get(dynatraceSecretName, metav1.GetOptions{}) if err != nil { return nil, err diff --git a/pkg/lib/dynatrace_events.go b/pkg/lib/dynatrace_events.go index 425b76867..1e1f6aa05 100644 --- a/pkg/lib/dynatrace_events.go +++ b/pkg/lib/dynatrace_events.go @@ -5,8 +5,8 @@ import ( ) // Sends an event to the Dynatrace events API -func (dt *DynatraceHelper) SendEvent(dtEvent interface{}) { - dt.Logger.Info("Sending event to Dynatrace API") +func (dt *DynatraceHelper) SendEvent(dtEvent interface{}, dtCreds string) { + dt.Logger.Info("Sending event to Dynatrace API using dtCreds: " + dtCreds) jsonString, err := json.Marshal(dtEvent) @@ -15,7 +15,7 @@ func (dt *DynatraceHelper) SendEvent(dtEvent interface{}) { return } - body, err := dt.sendDynatraceAPIRequest("/api/v1/events", "POST", string(jsonString)) + body, err := dt.sendDynatraceAPIRequest(dtCreds, "/api/v1/events", "POST", string(jsonString)) if err != nil { dt.Logger.Error("Failed sending Dynatrace API request: " + err.Error()) diff --git a/pkg/lib/metric_events.go b/pkg/lib/metric_events.go index 9cfc70b5c..2cfc010e3 100644 --- a/pkg/lib/metric_events.go +++ b/pkg/lib/metric_events.go @@ -4,12 +4,14 @@ import ( "encoding/json" "errors" "fmt" - "os" "regexp" "strconv" "strings" "github.com/ghodss/yaml" + + "github.com/keptn-contrib/dynatrace-service/pkg/common" + configutils "github.com/keptn/go-utils/pkg/api/utils" keptn "github.com/keptn/go-utils/pkg/lib" ) @@ -81,7 +83,7 @@ func (dt *DynatraceHelper) CreateMetricEvents(project string, stage string, serv mePayload, _ = json.Marshal(event) } - resp, err := dt.sendDynatraceAPIRequest(apiURL, apiMethod, string(mePayload)) + resp, err := dt.sendDynatraceAPIRequest("", apiURL, apiMethod, string(mePayload)) dt.Logger.Debug(resp) if err != nil { dt.Logger.Error("Could not create metric event " + newMetricEvent.Name + ": " + err.Error() + ": " + resp) @@ -100,7 +102,7 @@ func (dt *DynatraceHelper) CreateMetricEvents(project string, stage string, serv } func (dt *DynatraceHelper) GetMetricEvent(eventKey string) (*MetricEvent, error) { - res, err := dt.sendDynatraceAPIRequest("/api/config/v1/anomalyDetection/metricEvents", "GET", "") + res, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/anomalyDetection/metricEvents", "GET", "") if err != nil { dt.Logger.Error("Could not retrieve list of existing Dynatrace metric events: " + err.Error()) return nil, err @@ -116,7 +118,7 @@ func (dt *DynatraceHelper) GetMetricEvent(eventKey string) (*MetricEvent, error) for _, metricEvent := range dtMetricEvents.Values { if metricEvent.Name == eventKey { - res, err = dt.sendDynatraceAPIRequest("/api/config/v1/anomalyDetection/metricEvents/"+metricEvent.ID, "GET", "") + res, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/anomalyDetection/metricEvents/"+metricEvent.ID, "GET", "") if err != nil { dt.Logger.Error("Could not get existing metric event " + eventKey + ": " + err.Error()) return nil, err @@ -133,7 +135,7 @@ func (dt *DynatraceHelper) GetMetricEvent(eventKey string) (*MetricEvent, error) } func (dt *DynatraceHelper) DeleteExistingMetricEvent(eventKey string) error { - res, err := dt.sendDynatraceAPIRequest("/api/config/v1/anomalyDetection/metricEvents", "GET", "") + res, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/anomalyDetection/metricEvents", "GET", "") if err != nil { dt.Logger.Error("Could not retrieve list of existing Dynatrace metric events: " + err.Error()) return err @@ -149,7 +151,7 @@ func (dt *DynatraceHelper) DeleteExistingMetricEvent(eventKey string) error { for _, metricEvent := range dtMetricEvents.Values { if metricEvent.Name == eventKey { - res, err = dt.sendDynatraceAPIRequest("/api/config/v1/anomalyDetection/metricEvents/"+metricEvent.ID, "DELETE", "") + res, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/anomalyDetection/metricEvents/"+metricEvent.ID, "DELETE", "") if err != nil { dt.Logger.Error("Could not delete existing metric event " + eventKey + ": " + err.Error()) return err @@ -159,15 +161,8 @@ func (dt *DynatraceHelper) DeleteExistingMetricEvent(eventKey string) error { return nil } -func getConfigurationServiceURL() string { - if os.Getenv("CONFIGURATION_SERVICE_URL") != "" { - return os.Getenv("CONFIGURATION_SERVICE_URL") - } - return "configuration-service.keptn.svc.cluster.local:8080" -} - func retrieveSLOs(project string, stage string, service string) (*keptn.ServiceLevelObjectives, error) { - resourceHandler := configutils.NewResourceHandler(getConfigurationServiceURL()) + resourceHandler := configutils.NewResourceHandler(common.GetConfigurationServiceURL()) resource, err := resourceHandler.GetServiceResource(project, stage, service, "slo.yaml") if err != nil || resource.ResourceContent == "" { diff --git a/pkg/lib/problem_notifications.go b/pkg/lib/problem_notifications.go index 6e0c0c1bb..d565375f9 100644 --- a/pkg/lib/problem_notifications.go +++ b/pkg/lib/problem_notifications.go @@ -4,6 +4,7 @@ import ( "encoding/json" "strings" + "github.com/keptn-contrib/dynatrace-service/pkg/common" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) @@ -16,7 +17,7 @@ func (dt *DynatraceHelper) EnsureProblemNotificationsAreSetUp() error { return err } - response, err := dt.sendDynatraceAPIRequest("/api/config/v1/notifications", "GET", "") + response, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/notifications", "GET", "") existingNotifications := &DTAPIListResponse{} @@ -27,17 +28,12 @@ func (dt *DynatraceHelper) EnsureProblemNotificationsAreSetUp() error { for _, notification := range existingNotifications.Values { if notification.Name == "Keptn Problem Notification" { - _, _ = dt.sendDynatraceAPIRequest("/api/config/v1/notifications/"+notification.ID, "DELETE", "") + _, _ = dt.sendDynatraceAPIRequest("", "/api/config/v1/notifications/"+notification.ID, "DELETE", "") } } problemNotification := PROBLEM_NOTIFICATION_PAYLOAD - keptnDomainCM, err := dt.KubeApi.CoreV1().ConfigMaps("keptn").Get("keptn-domain", metav1.GetOptions{}) - if err != nil { - dt.Logger.Error("Could not retrieve keptn-domain ConfigMap: " + err.Error()) - } - - keptnDomain := keptnDomainCM.Data["app_domain"] + keptnDomain, _ := common.GetKeptnDomain() problemNotification = strings.ReplaceAll(problemNotification, "$KEPTN_DNS", "https://api.keptn."+keptnDomain) @@ -52,7 +48,7 @@ func (dt *DynatraceHelper) EnsureProblemNotificationsAreSetUp() error { problemNotification = strings.ReplaceAll(problemNotification, "$ALERTING_PROFILE_ID", alertingProfileId) - _, err = dt.sendDynatraceAPIRequest("/api/config/v1/notifications", "POST", problemNotification) + _, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/notifications", "POST", problemNotification) if err != nil { dt.Logger.Error("could not set up problem notification: " + err.Error()) return err @@ -62,7 +58,7 @@ func (dt *DynatraceHelper) EnsureProblemNotificationsAreSetUp() error { func (dt *DynatraceHelper) setupAlertingProfile() (string, error) { dt.Logger.Info("Checking Keptn alerting profile availability") - response, err := dt.sendDynatraceAPIRequest("/api/config/v1/alertingProfiles", "GET", "") + response, err := dt.sendDynatraceAPIRequest("", "/api/config/v1/alertingProfiles", "GET", "") existingAlertingProfiles := &DTAPIListResponse{} @@ -84,7 +80,7 @@ func (dt *DynatraceHelper) setupAlertingProfile() (string, error) { alertingProfilePayload, _ := json.Marshal(alertingProfile) - response, err = dt.sendDynatraceAPIRequest("/api/config/v1/alertingProfiles", "POST", string(alertingProfilePayload)) + response, err = dt.sendDynatraceAPIRequest("", "/api/config/v1/alertingProfiles", "POST", string(alertingProfilePayload)) if err != nil { return "", err diff --git a/test-events/configuration-change.http b/test-events/configuration-change.http new file mode 100644 index 000000000..f527ff869 --- /dev/null +++ b/test-events/configuration-change.http @@ -0,0 +1,37 @@ +# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection) or +# paste cURL into the file and request will be converted to HTTP Request format. +# +# Following HTTP Request Live Templates are available: +# * 'gtrp' and 'gtr' create a GET request with or without query parameters; +# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body; +# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data); + +POST http://localhost:8080/ +Accept: application/json +Cache-Control: no-cache +Content-Type: application/cloudevents+json + +{ + "type": "sh.keptn.event.configuration.change", + "contenttype": "application/json", + "specversion": "0.2", + "source": "test-event", + "id": "f2b878d3-03c0-4e8f-bc3f-454bc1b3d79d", + "time": "2019-06-07T07:02:15.64489Z", + "shkeptncontext": "08735340-6f9e-4b32-97ff-3b6c292bc509", + "data": { + "project": "sockshop", + "stage": "dev", + "service": "carts", + "valuesCanary": { + "image": "docker.io/keptnexamples/carts:0.10.1" + }, + "labels": { + "testid": "12345", + "buildnr": "build17", + "runby": "JohnDoe" + } + } +} + +### \ No newline at end of file diff --git a/test-events/deployment-finished.http b/test-events/deployment-finished.http new file mode 100644 index 000000000..1908b933b --- /dev/null +++ b/test-events/deployment-finished.http @@ -0,0 +1,41 @@ +# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection) or +# paste cURL into the file and request will be converted to HTTP Request format. +# +# Following HTTP Request Live Templates are available: +# * 'gtrp' and 'gtr' create a GET request with or without query parameters; +# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body; +# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data); + +POST http://localhost:8080/ +Accept: application/json +Cache-Control: no-cache +Content-Type: application/cloudevents+json + +{ + "type": "sh.keptn.events.deployment-finished", + "contenttype": "application/json", + "specversion": "0.2", + "source": "test-event", + "id": "f2b878d3-03c0-4e8f-bc3f-454bc1b3d79d", + "time": "2019-06-07T07:02:15.64489Z", + "shkeptncontext": "08735340-6f9e-4b32-97ff-3b6c292bc509", + "data": { + "project": "sockshop", + "stage": "dev", + "service": "carts", + "testStrategy": "performance", + "deploymentStrategy": "direct", + "tag": "0.10.1", + "image": "docker.io/keptnexamples/carts", + "labels": { + "testid": "12345", + "buildnr": "build17", + "runby": "JohnDoe", + "environment" : "testenvironment", + "ciBackLink" : "http://myjenkinsserver/job/12345" + }, + "deploymentURILocal": "http://carts.sockshop-staging.svc.cluster.local", + "deploymentURIPublic": "https://carts.sockshop-staging.my-domain.com" + } +} +### diff --git a/test-events/evaluation-done.http b/test-events/evaluation-done.http new file mode 100644 index 000000000..822328979 --- /dev/null +++ b/test-events/evaluation-done.http @@ -0,0 +1,72 @@ +# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection) or +# paste cURL into the file and request will be converted to HTTP Request format. +# +# Following HTTP Request Live Templates are available: +# * 'gtrp' and 'gtr' create a GET request with or without query parameters; +# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body; +# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data); + +POST http://localhost:8080/ +Accept: application/json +Cache-Control: no-cache +Content-Type: application/cloudevents+json + +{ + "type": "sh.keptn.events.evaluation-done", + "contenttype": "application/json", + "specversion": "0.2", + "source": "test-event", + "data":{ + "deploymentstrategy":"direct", + "evaluationdetails":{ + "indicatorResults":[ + { + "score":0, + "status":"failed", + "targets":[ + { + "criteria":"<=800", + "targetValue":800, + "violated":true + }, + { + "criteria":"<=+10%", + "targetValue":549.1967956487127, + "violated":true + }, + { + "criteria":"<600", + "targetValue":600, + "violated":true + } + ], + "value":{ + "metric":"response_time_p95", + "success":true, + "value":1002.6278552658177 + } + } + ], + "result":"fail", + "score":73.07692307692307, + "sloFileContent":"LS0tDQpzcGVjX3ZlcnNpb246ICcxLjAnDQpjb21wYXJpc29uOg0KICBjb21wYXJlX3dpdGg6ICJzaW5nbGVfcmVzdWx0Ig0KICBpbmNsdWRlX3Jlc3VsdF93aXRoX3Njb3JlOiAicGFzcyINCiAgYWdncmVnYXRlX2Z1bmN0aW9uOiBhdmcNCm9iamVjdGl2ZXM6DQogIC0gc2xpOiByZXNwb25zZV90aW1lX3A5NQ0KICAgIHBhc3M6ICAgICAgICAjIHBhc3MgaWYgKHJlbGF0aXZlIGNoYW5nZSA8PSAxMCUgQU5EIGFic29sdXRlIHZhbHVlIGlzIDwgNTAwKQ0KICAgICAgLSBjcml0ZXJpYToNCiAgICAgICAgICAtICI8PSsxMCUiICMgcmVsYXRpdmUgdmFsdWVzIHJlcXVpcmUgYSBwcmVmaXhlZCBzaWduIChwbHVzIG9yIG1pbnVzKQ0KICAgICAgICAgIC0gIjw2MDAiICAgIyBhYnNvbHV0ZSB2YWx1ZXMgb25seSByZXF1aXJlIGEgbG9naWNhbCBvcGVyYXRvcg0KICAgIHdhcm5pbmc6ICAgICAjIGlmIHRoZSByZXNwb25zZSB0aW1lIGlzIGJlbG93IDgwMG1zLCB0aGUgcmVzdWx0IHNob3VsZCBiZSBhIHdhcm5pbmcNCiAgICAgIC0gY3JpdGVyaWE6DQogICAgICAgICAgLSAiPD04MDAiDQp0b3RhbF9zY29yZToNCiAgcGFzczogIjkwJSINCiAgd2FybmluZzogNzUl", + "timeEnd":"2019-11-18T11:29:36Z", + "timeStart":"2019-11-18T11:21:06Z" + }, + "project":"sockshop", + "result":"fail", + "service":"carts", + "stage":"dev", + "teststrategy":"performance", + "labels": { + "testid": "12345", + "buildnr": "build17", + "runby": "JohnDoe" + } + }, + "id":"1b7cd584-320e-4ef0-8522-8a817263fdab", + "time":"2019-11-18T11:30:45.340Z", + "shkeptncontext":"60077081-f902-4407-bc15-7c70be41a836" +} + +### \ No newline at end of file diff --git a/test-events/get-sli.http b/test-events/get-sli.http new file mode 100644 index 000000000..8f9279e5d --- /dev/null +++ b/test-events/get-sli.http @@ -0,0 +1,43 @@ +# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection) or +# paste cURL into the file and request will be converted to HTTP Request format. +# +# Following HTTP Request Live Templates are available: +# * 'gtrp' and 'gtr' create a GET request with or without query parameters; +# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body; +# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data); + +POST http://localhost:8080/ +Accept: application/json +Cache-Control: no-cache +Content-Type: application/cloudevents+json + +{ + "type": "sh.keptn.internal.event.get-sli", + "contenttype": "application/json", + "specversion": "0.2", + "source": "test-event", + "data": { + "customFilters": [ + ], + "deploymentstrategy": "direct", + "end": "2019-11-19T09:26:53Z", + "indicators": [ + "response_time_p95" + ], + "project": "sockshop", + "service": "carts", + "sliProvider": "dynatrace", + "stage": "dev", + "start": "2019-11-19T09:18:14Z", + "teststrategy": "performance", + "labels": { + "testid": "12345", + "buildnr": "build17", + "runby": "JohnDoe" + } + }, + "id": "432b41e2-e50e-4d55-a4ca-0076df5b0b29", + "time": "2019-11-19T09:26:54.282Z", + "shkeptncontext": "cc42042e-9d25-48cb-a0df-ad8c2e30b6d7" +} +### diff --git a/test-events/tests-finished.http b/test-events/tests-finished.http new file mode 100644 index 000000000..7a19c3412 --- /dev/null +++ b/test-events/tests-finished.http @@ -0,0 +1,39 @@ +# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection) or +# paste cURL into the file and request will be converted to HTTP Request format. +# +# Following HTTP Request Live Templates are available: +# * 'gtrp' and 'gtr' create a GET request with or without query parameters; +# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body; +# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data); + +POST http://localhost:8080/ +Accept: application/json +Cache-Control: no-cache +Content-Type: application/cloudevents+json + +{ + "type": "sh.keptn.events.tests-finished", + "contenttype": "application/json", + "specversion": "0.2", + "source": "test-event", + "id": "f2b878d3-03c0-4e8f-bc3f-454bc1b3d79d", + "time": "2019-06-07T07:02:15.64489Z", + "shkeptncontext": "08735340-6f9e-4b32-97ff-3b6c292bc509", + "data": { + "project": "sockshop", + "stage": "dev", + "service": "carts", + "testStrategy": "performance", + "deploymentStrategy": "direct", + "start": "2019-09-01 12:00:00", + "end": "2019-09-01 12:05:00", + "labels": { + "testid": "12345", + "buildnr": "build17", + "runby": "JohnDoe" + }, + "result": "pass" + } +} + +### \ No newline at end of file