docs: Improve troubleshooting content for evaluation failed (#734)

* Reformat and improve content Signed-off-by: Arthur Pitman <[email protected]> * Fix semantic PR Signed-off-by: Arthur Pitman <[email protected]>
keptn-contrib · Mar 9, 2022 · 4fbd86a · 4fbd86a
1 parent 0c4208c
commit 4fbd86a
Show file tree

Hide file tree

Showing 6 changed files with 52 additions and 26 deletions.
diff --git a/documentation/images/could-not-find-a-matching-dashboard.png b/documentation/images/could-not-find-a-matching-dashboard.png
diff --git a/documentation/images/dashboard-not-found.png b/documentation/images/dashboard-not-found.png
diff --git a/documentation/images/dynatrace-metrics-api-returned-zero-data-points.png b/documentation/images/dynatrace-metrics-api-returned-zero-data-points.png
diff --git a/documentation/images/get-sli-finished-event-payload.png b/documentation/images/get-sli-finished-event-payload.png
diff --git a/documentation/images/no-evaluation-performed-by-lighthouse.png b/documentation/images/no-evaluation-performed-by-lighthouse.png
diff --git a/documentation/troubleshooting.md b/documentation/troubleshooting.md
@@ -39,46 +39,72 @@ Likely cause:
 - SLIs and SLOs should be defined using YAML files but no SLOs have been defined i.e. no `slo.yaml` file can be found
 
 Suggested solution:
-- If SLIs and SLOs should be defined using YAML files, define SLOs by creating a `slo.yaml` file, or
-- If SLIs and SLOs should be sourced from a dashboard, add a [`dashboard` entry to the `dynatrace/dynatrace.conf.yaml` configuration file](dynatrace-conf-yaml-file.md#dashboard-sli-mode-configuration-dashboard)
 
-## Evaluation fails: Dynatrace Metrics API returned zero data points
+
+## Evaluation fails: No evaluation performed by Lighthouse because SLI failed
 
 Evaluation fails and the shipyard-controller reports:
-> `no evaluation performed by lighthouse because SLI failed with message Dynatrace Metrics API returned zero data points...`
+> `no evaluation performed by lighthouse because SLI failed....`
 
-![Dynatrace Metrics API returned zero data points](images/dynatrace-metrics-api-returned-zero-data-points.png)
+![No evaluation performed by Lighthouse](images/no-evaluation-performed-by-lighthouse.png)
 
-Likely cause:
-- No data is available during the evaluation timeframe
+In this case the Lighthouse performed no evaluation because the received `sh.keptn.event.get-sli.finished` event had a `result` of `fail`. The root cause can be identified from the `message` field of this event.
 
-Suggested solution:
-- Check the availability of data for the evaluation timeframe, e.g. using the Data Explorer in the Dynatrace tenant
+In instances where this is related to a specific SLI, check the `message` field of the individual SLI result in the within `indicatorValues`. This can be viewed in Keptn UI by expanding the `sh.keptn.event.get-sli.finished` event and viewing its payload:
 
-## Evaluation fails: Could not query Dynatrace dashboard for SLIs: ...could not find a matching dashboard name
+![View `sh.keptn.event.get-sli.finished` payload](images/get-sli-finished-event-payload.png)
 
-Evaluation fails and the shipyard-controller reports:
-> `could not query Dynatrace dashboard for SLIs: error while processing dashboard config '' - could not find a matching dashboard name - e.g. KQG;project=<project>;service=<service>;stage=<stage>`
+The following subsections detail some common messages as well as likely causes and solutions:
 
-![Could not find a matching dashboard](images/could-not-find-a-matching-dashboard.png)
+### Message: `Could not retrieve any SLI results` 
 
-Likely cause:
-- Dynatrace-service is configured to query a dashboard for SLIs (i.e. `dynatrace/dynatrace.conf.yaml` contains `dashboard: query`) but no dashboard named with the pattern `KQG;project=<project>;service=<service>;stage=<stage>` is available
+**Likely cause**
 
-Suggested solution:
-- If SLIs and SLOs should be sourced from a dashboard, create a dashboard named with the pattern `KQG;project=<project>;service=<service>;stage=<stage>` in the Dynatrace tenant, or
-- If SLIs and SLOs should be defined using YAML files, remove the [`dashboard: query` entry from `dynatrace/dynatrace.conf.yaml`](dynatrace-conf-yaml-file.md#dashboard-sli-mode-configuration-dashboard)
+- SLIs and SLOs should be defined using YAML files but no SLOs have been defined, i.e. no `slo.yaml` file can be found
 
-## Evaluation fails: Could not query Dynatrace dashboard for SLIs: ...Dashboard "..." not found
+**Suggested solution**
 
-Evaluation fails and the shipyard-controller reports:
-> `could not query Dynatrace dashboard for SLIs: error while processing dashboard config '12345678-1234-1234-1234-12345678abcd' - Dynatrace API error (404): Dashboard 12345678-1234-1234-1234-12345678abcd not found`
+- If SLIs and SLOs should be defined using YAML files, define SLOs by creating a `slo.yaml` file, or
+- If SLIs and SLOs should be sourced from a dashboard, add a [`dashboard` entry to the `dynatrace/dynatrace.conf.yaml` configuration file](dynatrace-conf-yaml-file.md#dashboard-sli-mode-configuration-dashboard)
 
-![Dashboard not found](images/dashboard-not-found.png)
+### Message: `Metrics API v2 returned zero data points`
+
+**Likely cause**
+
+- No data is available for an SLI during the evaluation timeframe
+
+**Suggested solution**
+
+- Check the availability of data for the associated SLIs during the evaluation timeframe, e.g. using the Data Explorer in the Dynatrace tenant
+
+### Message: `Metrics API v2 returned more than one data point`
+
+**Likely cause**
+
+- Multiple data points are returned for an SLI during the evaluation timeframe where only one is expected
+
+**Suggested solution**
+
+- Ensure the SLI only returns a single data point by e.g. including a `merge` or `splitBy` transformation in the query
+
+### Message: `could not query Dynatrace dashboard for SLIs: error while processing dashboard config '12345678-1234-1234-1234-12345678abcd' - Dynatrace API error (404): Dashboard 12345678-1234-1234-1234-12345678abcd not found`
+
+**Likely cause**
 
-Likely cause:
 - Dynatrace-service is configured to query a specific dashboard for SLIs (i.e. [`dynatrace/dynatrace.conf.yaml` contains `dashboard: <dashboard-id>`](dynatrace-conf-yaml-file.md#dashboard-sli-mode-configuration-dashboard)) but no dashboard with the specified ID is available
 
-Suggested solution:
-- If SLIs and SLOs should be sourced from a dashboard, ensure `dynatrace/dynatrace.conf.yaml` contains a `dashboard: <dashboard-id>` entry with the correct dashboard ID, or
-- If SLIs and SLOs should be defined using YAML files, remove the `dashboard: <dashboard-id>` entry from `dynatrace/dynatrace.conf.yaml`
+**Suggested solution**
+
+ - If SLIs and SLOs should be sourced from a dashboard, ensure `dynatrace/dynatrace.conf.yaml` contains a `dashboard: <dashboard-id>` entry with the correct dashboard ID, or
+ - If SLIs and SLOs should be defined using YAML files, remove the `dashboard: <dashboard-id>` entry from `dynatrace/dynatrace.conf.yaml`
+
+### Message: `could not query Dynatrace dashboard for SLIs: error while processing dashboard config '' - could not find a matching dashboard name - e.g. KQG;project=<project>;service=<service>;stage=<stage>`
+
+**Likely cause**
+
+- Dynatrace-service is configured to query a dashboard for SLIs (i.e. `dynatrace/dynatrace.conf.yaml` contains `dashboard: query`) but no dashboard named with the pattern `KQG;project=<project>;service=<service>;stage=<stage>` is available
+
+**Suggested solution**
+
+- If SLIs and SLOs should be sourced from a dashboard, create a dashboard named with the pattern `KQG;project=<project>;service=<service>;stage=<stage>` in the Dynatrace tenant, or
+- If SLIs and SLOs should be defined using YAML files, remove the [`dashboard: query` entry from `dynatrace/dynatrace.conf.yaml`](dynatrace-conf-yaml-file.md#dashboard-sli-mode-configuration-dashboard)