Skip to content

Commit

Permalink
docs: Add example for OTel eBPF profiler (#3695)
Browse files Browse the repository at this point in the history
* feat(examples): Add examples for OTel eBPF profiler
  • Loading branch information
marcsanmi authored Nov 20, 2024
1 parent 4496fe2 commit ed1bba5
Show file tree
Hide file tree
Showing 11 changed files with 467 additions and 0 deletions.
82 changes: 82 additions & 0 deletions examples/grafana-agent-auto-instrumentation/ebpf-otel/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# OpenTelemetry eBPF profiler examples

**⚠️ Important: Linux-only Support**
This example can only be run on Linux systems (amd64/arm64) as it relies on eBPF technology which is specific to the Linux kernel. The profiler requires privileged access to system resources.
For more details refer to the OpenTelemetry ebpf profiler [docs](https://github.com/open-telemetry/opentelemetry-ebpf-profiler).

These examples demonstrate:
1. OpenTelemetry eBPF profiler collecting system-wide profiles
2. OpenTelemetry Collector receiving and processing the data from the profiler
3. Pyroscope receiving and visualizing the profiles via Grafana

## Prerequisites
**⚠️ Important:** Since the [profiler image](https://hub.docker.com/r/otel/opentelemetry-ebpf-profiler-dev) is not publicly available yet, you need to build the profiler binary first.

Follow the build instructions:

1. Build the profiler binary:

```bash
# Clone the repository
git clone https://github.com/open-telemetry/opentelemetry-ebpf-profiler
cd opentelemetry-ebpf-profiler

# Build the environment
make docker-image

# Build the profiler binary
make agent
```

2. Copy the built binary to the example directory:
```bash
# Copy the ebpf-profiler binary to the example directory
cp ebpf-profiler /path/to/example/directory/
```
**Note:** The following examples will consider that an `ebpf-profiler` binary is already existing on each example root directory.

For more details, please refer to opentelemetry-ebpf-profiler [repository](https://github.com/open-telemetry/opentelemetry-ebpf-profiler)

## Docker example
1. Start the environment:

```bash
# Start all services
docker-compose up --build

# To clean up
docker-compose down
```
2. Access the UI:
```bash
# Access Grafana
http://localhost:3000
```

## Kubernetes example

1. Build and prepare the profiler image:

```bash
# Build the image with the binary
docker build -t test-ebpf-profiler:latest .

# Make the image available if necessary. e.g in Minikube
minikube image load test-ebpf-profiler:latest
```
2. Deploy to Kubernetes:
```bash
# Apply the manifests
kubectl apply -f kubernetes/

# Clean up
kubectl delete -f kubernetes/
```
3. Access the UI:
```bash
# Port forward Grafana
kubectl port-forward svc/grafana-service 3000:3000

# Access Grafana
http://localhost:3000
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM ubuntu:22.04

RUN apt-get update && \
apt-get install -y linux-headers-generic && \
rm -rf /var/lib/apt/lists/*

COPY ebpf-profiler /usr/local/bin/

ENTRYPOINT ["/usr/local/bin/ebpf-profiler", "-collection-agent", "otel-collector:4317", "-no-kernel-version-check", "-disable-tls"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
receivers:
otlp:
protocols:
grpc:
endpoint: "0.0.0.0:4317"
http:

processors:
batch:

exporters:
debug:
verbosity: detailed
otlp:
endpoint: "http://pyroscope:4040"
tls:
insecure: true

service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [debug]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [debug]
logs:
receivers: [otlp]
processors: [batch]
exporters: [debug]
profiles:
receivers: [otlp]
exporters: [otlp]

telemetry:
logs:
level: debug
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
version: '3'
services:
otel-collector:
image: otel/opentelemetry-collector-contrib:latest
command: ["--config=/etc/otel-collector-config.yaml", "--feature-gates=service.profilesSupport"]
volumes:
- ./config/otel-collector-config.yaml:/etc/otel-collector-config.yaml
ports:
- "4317:4317"
- "4318:4318"
networks:
- otel-net
depends_on:
- pyroscope

otel-ebpf-profiler:
build: .
hostname: ebpf-profiler
privileged: true
pid: "host"
volumes:
- /sys/kernel/debug:/sys/kernel/debug
- /sys/fs/cgroup:/sys/fs/cgroup
- /proc:/proc
networks:
- otel-net
depends_on:
- otel-collector

pyroscope:
image: grafana/pyroscope:latest
command: ["-self-profiling.disable-push=true"]
ports:
- "4040:4040"
networks:
- otel-net

grafana:
image: grafana/grafana:latest
environment:
- GF_INSTALL_PLUGINS=grafana-pyroscope-app
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
- GF_AUTH_DISABLE_LOGIN_FORM=true
volumes:
- ./grafana-provisioning:/etc/grafana/provisioning
ports:
- "3000:3000"
networks:
- otel-net

networks:
otel-net:
driver: bridge
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
apiVersion: 1
datasources:
- uid: local-pyroscope
type: grafana-pyroscope-datasource
name: Pyroscope
url: http://pyroscope:4040
jsonData:
keepCookies: [pyroscope_git_session]
# Uncomment these if using with Grafana Cloud
# basicAuth: true
# basicAuthUser: '123456'
# secureJsonData:
# basicAuthPassword: PASSWORD
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM ubuntu:22.04

RUN apt-get update && \
apt-get install -y linux-headers-generic && \
rm -rf /var/lib/apt/lists/*

COPY ebpf-profiler /usr/local/bin/

ENTRYPOINT ["/usr/local/bin/ebpf-profiler", "-collection-agent", "otel-collector-service:4317", "-no-kernel-version-check", "-disable-tls"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: cpu-stress
spec:
replicas: 1
selector:
matchLabels:
app: cpu-stress
template:
metadata:
labels:
app: cpu-stress
spec:
containers:
- name: stress
image: polinux/stress
command: ["stress"]
args: ["--cpu", "2"]
resources:
limits:
cpu: "2"
requests:
cpu: "1"
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana
spec:
selector:
matchLabels:
app: grafana
template:
metadata:
labels:
app: grafana
spec:
containers:
- name: grafana
image: grafana/grafana:latest
ports:
- containerPort: 3000
env:
- name: GF_INSTALL_PLUGINS
value: grafana-pyroscope-app
- name: GF_AUTH_ANONYMOUS_ENABLED
value: "true"
- name: GF_AUTH_ANONYMOUS_ORG_ROLE
value: Admin
- name: GF_AUTH_DISABLE_LOGIN_FORM
value: "true"
volumeMounts:
- name: grafana-provisioning
mountPath: /etc/grafana/provisioning
volumes:
- name: grafana-provisioning
configMap:
name: grafana-provisioning
items:
- key: datasources
path: datasources/datasources.yaml
- key: plugins
path: plugins/plugins.yaml
---
apiVersion: v1
kind: Service
metadata:
name: grafana-service
spec:
selector:
app: grafana
ports:
- protocol: TCP
port: 3000
targetPort: 3000
---
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-provisioning
data:
"datasources": |
apiVersion: 1
datasources:
- uid: local-pyroscope
type: grafana-pyroscope-datasource
name: Pyroscope
url: http://pyroscope-service:4040
jsonData:
keepCookies: [pyroscope_git_session]
"plugins": |
apiVersion: 1
apps:
- type: grafana-pyroscope-app
jsonData:
backendUrl: http://pyroscope-service:4040
secureJsonData:
Loading

0 comments on commit ed1bba5

Please sign in to comment.