Is it Observable

Episode : Target Allocator

This repository contains the files utilized during the tutorial presented in the dedicated IsItObservable episode related to the episode How to easily collect Prometheus metrics with OpenTelemetry.

this tutorial will also utilize the OpenTelemetry Operator with:

• the OpenTelemetry Demo
• the hipster-shop
• istio All the observability data generated by the environment would be sent to Dynatrace.

Prerequisite

The following tools need to be install on your machine :

• jq
• kubectl
• git
• gcloud ( if you are using GKE)
• Helm

1.Create a Google Cloud Platform Project

PROJECT_ID="<your-project-id>"
gcloud services enable container.googleapis.com --project ${PROJECT_ID}
gcloud services enable monitoring.googleapis.com \
cloudtrace.googleapis.com \
clouddebugger.googleapis.com \
cloudprofiler.googleapis.com \
--project ${PROJECT_ID}

2.Create a GKE cluster

ZONE=europe-west3-a
NAME=isitobservable-targetallocator
gcloud container clusters create ${NAME} --zone=${ZONE} --machine-type=e2-standard-4 --num-nodes=2

Getting started

Dynatrace Tenant

1. Dynatrace Tenant - start a trial

If you don't have any Dynatrace tenant , then i suggest to create a trial using the following link : Dynatrace Trial Once you have your Tenant save the Dynatrace tenant url in the variable DT_TENANT_URL (for example : https://dedededfrf.live.dynatrace.com)

DT_TENANT_URL=<YOUR TENANT Host>

2. Create the Dynatrace API Tokens

The dynatrace operator will require to have several tokens:

• Token to deploy and configure the various components
• Token to ingest metrics and Traces

Operator Token

One for the operator having the following scope:

• Create ActiveGate tokens
• Read entities
• Read Settings
• Write Settings
• Access problem and event feed, metrics and topology
• Read configuration
• Write configuration
• Paas integration - installer downloader

Save the value of the token . We will use it later to store in a k8S secret

API_TOKEN=<YOUR TOKEN VALUE>

Ingest data token

Create a Dynatrace token with the following scope:

• Ingest metrics (metrics.ingest)
• Ingest logs (logs.ingest)
• Ingest events (events.ingest)
• Ingest OpenTelemetry
• Read metrics

Save the value of the token . We will use it later to store in a k8S secret

DATA_INGEST_TOKEN=<YOUR TOKEN VALUE>

Istio

1. Download Istioctl

curl -L https://istio.io/downloadIstio | sh -

This command download the latest version of istio ( in our case istio 1.18.2) compatible with our operating system. 2. Add istioctl to you PATH

cd istio-1.20.1

this directory contains samples with addons . We will refer to it later.

export PATH=$PWD/bin:$PATH

4.Clone Github repo

git clone https://github.com/isItObservable/target-allocator
cd target-allocator

Deploy most of the components

The application will deploy the entire environment:

chmod 777 deployment.sh
./deployment.sh  --clustername "${NAME}" --dturl "${DT_TENANT_URL}" --dtingesttoken "${DATA_INGEST_TOKEN}" --dtoperatortoken "${API_TOKEN}"

Create a Collector with the target Allocator

kubectl apply -f  opentelemetry/openTelemetry-manifest_statefulset.yaml

Now let's have a look at the workload created by the Operator

Enable the Prometheus CR

In opentelemetry/openTelemetry-manifest_statefulset.yaml enable the Prometheus CR support , by changing :

targetAllocator:
   enabled: true   

to:

targetAllocator:
  enabled: true
  prometheusCR
     enabled: true

once the file updated, redeploy the Collector Statefulset:

kubectl apply -f  opentelemetry/openTelemetry-manifest_statefulset.yaml

Deploy Pod Monitors

Let's deploy PodMonitors and service Monitors:

kubectl apply -f istio/podmonitor.yaml
kubectl apply -f kepler/serviceMonitor.yaml