Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: create keptn metrics getting started #1375

Merged
merged 41 commits into from
May 11, 2023
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
80f05f9
docs: create metrics getting started guide
StackScribe May 4, 2023
86d4b2f
markdownlint-fix
StackScribe May 4, 2023
53b10d3
Copy in install
StackScribe May 5, 2023
33a2906
fix dead link in metrics-gs
StackScribe May 6, 2023
a0ecd62
Merge branch 'main' into 0504-metrics-gs
StackScribe May 6, 2023
caccf2d
updated syntax for KeptnMetricsProvider and KeptnMetric
StackScribe May 8, 2023
b04853f
Full install/enable/annotations
StackScribe May 8, 2023
ac986e6
Merge branch 'main' into 0504-metrics-gs
StackScribe May 9, 2023
651adc6
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
0ade565
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
4b006b4
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
3b83409
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
7c1fd19
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
c257b08
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
6cf7572
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
ba579d2
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
eae05ba
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
b7842bc
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 9, 2023
5e10ab0
Giovanni comments
StackScribe May 9, 2023
1429130
Merge branch '0504-metrics-gs' of github.com:StackScribe/lifecycle-to…
StackScribe May 9, 2023
1b9fe6c
Merge branch 'main' into 0504-metrics-gs
StackScribe May 11, 2023
8da2851
fix markdownlint issues
aepfli May 11, 2023
372277f
a bit of polishing
StackScribe May 11, 2023
6cb9f98
Merge branch 'main' of github.com:keptn/lifecycle-toolkit into 0504-m…
StackScribe May 11, 2023
4b60da8
Merge branch 'main' into 0504-metrics-gs
StackScribe May 11, 2023
7189c15
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 11, 2023
063ec52
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 11, 2023
805cf1f
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 11, 2023
0dc265b
Merge branch '0504-metrics-gs' of github.com:StackScribe/lifecycle-to…
StackScribe May 11, 2023
489b083
Giovanni's comments
StackScribe May 11, 2023
a111dee
Merge branch 'main' into 0504-metrics-gs
StackScribe May 11, 2023
2408885
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 11, 2023
6fc5982
markdownlint-fix
StackScribe May 11, 2023
c9093ff
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 11, 2023
2adafa4
Merge branch 'main' into 0504-metrics-gs
StackScribe May 11, 2023
7343221
HPA xref
StackScribe May 11, 2023
7ab5779
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 11, 2023
e49dfb9
fix xref structure
StackScribe May 11, 2023
9d11971
Update docs/content/en/docs/getting-started/metrics/_index.md
StackScribe May 11, 2023
6a641db
Merge branch '0504-metrics-gs' of github.com:StackScribe/lifecycle-to…
StackScribe May 11, 2023
ac3728f
more xref
StackScribe May 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
267 changes: 267 additions & 0 deletions docs/content/en/docs/getting-started/metrics/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,270 @@ description: Learn how Keptn metrics enhances your deployment
weight: 25
---

The Keptn metrics component of the Keptn Lifecycle Toolkit
allow you to define any type of metric
from multiple instances of any type of data source in your Kubernetes cluster.
You may have tools like Argo, Flux, KEDA, HPA, or Keptn
that need observability data to make automated decisions.
Whether a rollout is good, whether to scale up or down.
Your observability data may come
from multiple observability solutions --
Datadog, Dynatrace, Lightstep, Honeycomb, Splunk,
or data directly from your cloud provider such as AWS, Google, or Azure.

The Keptn Lifecycle Toolkit hooks directly into Kubernetes primitives
so minimal configuration is required.

The
[Kubernetes metric server](https://github.com/kubernetes-sigs/metrics-server)
requires that you maintain point-to-point integrations
from Argo Rollouts, Flux, KEDA, and HPA.
Each has plugins but it is difficult to maintain them,
especially if you are using multiple tools,
and multible observability platforms,
and multiple instance of some tools or observability platforms.
The Keptn Metrics Server unifies and standardizes access to this data.

## Using this exercise

This exercise is based on the
[simplenode-dev](https://github.com/keptn-sandbox/klt-on-k3s-with-argocd)
example.
You can clone that repo to access it locally
or just look at it for examples
as you implement the functionality "from scratch"
on your local Kubernetes deployment cluster.

The steps to implement metrics in an existing cluster are:

1. [Install the Keptn Lifecycle Toolkit](../../install/install)
StackScribe marked this conversation as resolved.
Show resolved Hide resolved
1. Configure metrics to use
- [Define metrics providers](#define-metrics-providers)
- [Define KeptnMetric information](#define-keptnmetric-information)
- [View available metrics](#view-available-metrics)

If you want to create your own cluster to run this exercise,
follow the instructions in [Installation](../../install).

See the
[Introducing Keptn Lifecycle Toolkit](https://youtu.be/449HAFYkUlY)
video for a demonstration of this exercise.

## Define metrics to use

You need to define the external observability platforms
from which you want to pull data
and then the specific data you want to pull.
This data is pulled and fetched continuously
at an interval you specify for each specific bit of data.
Data is available through the resource and through the data provider itself,
as well as the Kubernetes CLI.

### Define metrics providers

Populate a
[KeptnMetricsProvider](../../yaml-crd-ref/metricsprovider.md)
resource for each external observability platform you want to use.

For our example, we define two observability platforms:

- `dev-prometheus`
- `dev-dynatrace`

You can specify a virtually unlimited number of providers,
including multiple instances of each observability platform.
Each one must be assigned a unique name,
identified by the type of platform it is
and the URL.

> Note: The video and example application use an older syntax
of the `KeptnMetricsProvider` and `KeptnMetric` resources.
The syntax shown in this document is correct for v0.7.1 and later.

Definition of
[dev-prometheus](https://github.com/keptn-sandbox/klt-on-k3s-with-argocd/blob/main/simplenode-dev/keptn-prometheus-provider.yaml)
data source:

```yaml
kind: KeptnMetricsProvider
metadata:
name: dev-prometheus
namespace: simplenode-dev
spec:
type: prometheus
targetserver: "http://prometheus-k8s-monitoring-svc.cluster.local:9090"
```

Definition of the
[dev-dynatrace](https://github.com/keptn-sandbox/klt-on-k3s-with-argocd/blob/main/simplenode-dev/dynatrace-provider.yaml.tmp)
data source.
Note that the `dev-dynatrace` server is protected by a secret key
so that information is included in the provider definition:

```yaml
kind: KeptnMetricsProvider
metadata:
name: dev-dynatrace
namespace: simplenode-dev
spec:
type: dynatrace
targetServer: "https://hci34192.live.dynatrace.com"
secretKeyRef
name: dynatrace
key: DT_TOKEN
...
```

StackScribe marked this conversation as resolved.
Show resolved Hide resolved
### Define KeptnMetric information

The [KeptnMetric](../../yaml-crd-ref/metric.md) resource
defines the information you want to gather,
specified as a query for the particular observability platform
you are using.
You can define any type of metric from any data source.

In our example, we define two bits of information to retrieve:

- Number of CPUs, derived from the `dev-prometheus` data platform
- `availability` SLO, derived from the `dev-dynatrace` data platform

Each of these are configured to fetch data every 10 seconds
but you could configure a different `fetchIntervalSeconds` value
for each metric.

The
[keptn-metric.yaml](https://github.com/keptn-sandbox/klt-on-k3s-with-argocd/blob/main/simplenode-dev/keptn-metric.yaml)
file for our example looks like:

```yaml
apiVersion: metrics.keptn.sh/v1alpha2
kind: Keptnmetric
metadata:
name: available-cpus
namespace: simplenode-dev
spec:
provider:
name: dev-prometheus
query: "sum(kube_node_status_cvapacity{resources`cpu`})"
fetchIntervalSeconds: 10
...
apiVersion: metrics.keptn.sh/v1alpha2
kind: Keptnmetric
metadata:
name: availability-slo
namespace: simplenode-dev
spec:
provider:
name: dev-dynatrace
query: "func:slo.availability_simplenodeservice"
fetchIntervalSeconds: 10
...
StackScribe marked this conversation as resolved.
Show resolved Hide resolved
```

Note the following:

- Populate one YAML file per metric
then apply all of them.
- Each metric is assigned a unique `name`.
- The value of the `spec.provider.name` field
must correspond to the name assigned in a
the `metadata.name` field of a `KeptnMetricsProvider` resource.
- Information is fetched in on a continuous basis
at a rate specified by the value of the `spec.fetchIntervalSeconds` field.

### View available metrics

Use the following command to view
the metrics that are configured in your cluster.
This example displays the two metrics we configured above:

```shell
kubectl get KeptnMetrics -A
```

```shell
NAMESPACE NAME PROVIDER QUERY
simplenode-dev availability-slo dev-dynatrace func:slo.availability_simplenodeservice
simplenode-dev available-cpus dev-prometheus sum(kube_node_status_capacity{resource=`cpu`})
```

## Run the metrics

As soon as you define your `KeptnMetricsProvider` and `KeptnMetric` resources,
the Lifecycle Toolkit begins collecting the metrics you defined.
You do not need to do anything else.

## Observing the metrics

The metrics can be retrieved
through CRs and through the Kubernetes Metric API.

The syntax to retrieve metrics from the CR is:

```shell
kubectl get keptnmetrics.metrics.keptn.sh -n <namespace> <metric-name>
```

For example, the output for the `available-cpus` metric looks like:

```shell
$ kubectl get keptnmetrics.metrics.keptn.sh -n simplenode-dev available-cpus

NAME PROVIDER QUERY VALUE
cpu-throttling my-provider sum(kube_node_status_capacity{resource=`cpu`}) 6.000
```

The syntax to retrieve metrics through the Kubernetes API is:

```yaml
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta2/namespaces/<namespace>/keptnmetrics.metrics.sh/<metric-name>/<metric-name>"
```

For example, the output for the `available-cpus` looks like:

```yaml
$ kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta2/namespaces/simplenode-dev/keptnmetrics.metrics.sh/available-cpus/available-cpus"

{
"kind": "MetricValueList",
"apiVersion": "custom.metrics.k8s.io/v1beta2",
"metadata": {},
"items": [
{
"describedObject": {
"kind": "KeptnMetric",
"namespace": "simplenode-dev",
"name": "available-cpus",
"apiVersion": "metrics.keptn.sh/v1alpha2"
},
"metric": {
"name": "available-cpus",
"selector": {}
},
"timestamp": "2023-05-11T08:05:36Z",
"value": "6"
}
]
}
```

You can also display the graphics using a dashboard such as Grafana.

## Implementing autoscaling with HPA

The Kubernetes HorizontalPodAutoscaler (HPA)
uses metrics to provide autoscaling for the cluster.
HPA can retrieve KeptnMetrics and use it to implement HPA.
See Using the [HorizontalPodAutoscaler](../../implmenting/evaluatemetrics)
for detailed information.
StackScribe marked this conversation as resolved.
Show resolved Hide resolved

TODO: Link to HPA subsection after that content is merged

## Learn more

To learn more about the Keptn Metrics Server, see:

- Architecture:
[Keptn Metrics Operator](../../concepts/architecture/components/metrics-operator/)
- More information about implementing Keptn Metrics:
[Keptn Metrics](../../implementing/evaluatemetrics.md/)