-
Notifications
You must be signed in to change notification settings - Fork 626
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: Add example for OTel eBPF profiler (#3695)
* feat(examples): Add examples for OTel eBPF profiler
- Loading branch information
Showing
11 changed files
with
467 additions
and
0 deletions.
There are no files selected for viewing
82 changes: 82 additions & 0 deletions
82
examples/grafana-agent-auto-instrumentation/ebpf-otel/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
# OpenTelemetry eBPF profiler examples | ||
|
||
**⚠️ Important: Linux-only Support** | ||
This example can only be run on Linux systems (amd64/arm64) as it relies on eBPF technology which is specific to the Linux kernel. The profiler requires privileged access to system resources. | ||
For more details refer to the OpenTelemetry ebpf profiler [docs](https://github.com/open-telemetry/opentelemetry-ebpf-profiler). | ||
|
||
These examples demonstrate: | ||
1. OpenTelemetry eBPF profiler collecting system-wide profiles | ||
2. OpenTelemetry Collector receiving and processing the data from the profiler | ||
3. Pyroscope receiving and visualizing the profiles via Grafana | ||
|
||
## Prerequisites | ||
**⚠️ Important:** Since the [profiler image](https://hub.docker.com/r/otel/opentelemetry-ebpf-profiler-dev) is not publicly available yet, you need to build the profiler binary first. | ||
|
||
Follow the build instructions: | ||
|
||
1. Build the profiler binary: | ||
|
||
```bash | ||
# Clone the repository | ||
git clone https://github.com/open-telemetry/opentelemetry-ebpf-profiler | ||
cd opentelemetry-ebpf-profiler | ||
|
||
# Build the environment | ||
make docker-image | ||
|
||
# Build the profiler binary | ||
make agent | ||
``` | ||
|
||
2. Copy the built binary to the example directory: | ||
```bash | ||
# Copy the ebpf-profiler binary to the example directory | ||
cp ebpf-profiler /path/to/example/directory/ | ||
``` | ||
**Note:** The following examples will consider that an `ebpf-profiler` binary is already existing on each example root directory. | ||
|
||
For more details, please refer to opentelemetry-ebpf-profiler [repository](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) | ||
|
||
## Docker example | ||
1. Start the environment: | ||
|
||
```bash | ||
# Start all services | ||
docker-compose up --build | ||
|
||
# To clean up | ||
docker-compose down | ||
``` | ||
2. Access the UI: | ||
```bash | ||
# Access Grafana | ||
http://localhost:3000 | ||
``` | ||
|
||
## Kubernetes example | ||
|
||
1. Build and prepare the profiler image: | ||
|
||
```bash | ||
# Build the image with the binary | ||
docker build -t test-ebpf-profiler:latest . | ||
|
||
# Make the image available if necessary. e.g in Minikube | ||
minikube image load test-ebpf-profiler:latest | ||
``` | ||
2. Deploy to Kubernetes: | ||
```bash | ||
# Apply the manifests | ||
kubectl apply -f kubernetes/ | ||
|
||
# Clean up | ||
kubectl delete -f kubernetes/ | ||
``` | ||
3. Access the UI: | ||
```bash | ||
# Port forward Grafana | ||
kubectl port-forward svc/grafana-service 3000:3000 | ||
|
||
# Access Grafana | ||
http://localhost:3000 | ||
``` |
9 changes: 9 additions & 0 deletions
9
examples/grafana-agent-auto-instrumentation/ebpf-otel/docker/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
FROM ubuntu:22.04 | ||
|
||
RUN apt-get update && \ | ||
apt-get install -y linux-headers-generic && \ | ||
rm -rf /var/lib/apt/lists/* | ||
|
||
COPY ebpf-profiler /usr/local/bin/ | ||
|
||
ENTRYPOINT ["/usr/local/bin/ebpf-profiler", "-collection-agent", "otel-collector:4317", "-no-kernel-version-check", "-disable-tls"] |
39 changes: 39 additions & 0 deletions
39
...les/grafana-agent-auto-instrumentation/ebpf-otel/docker/config/otel-collector-config.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
receivers: | ||
otlp: | ||
protocols: | ||
grpc: | ||
endpoint: "0.0.0.0:4317" | ||
http: | ||
|
||
processors: | ||
batch: | ||
|
||
exporters: | ||
debug: | ||
verbosity: detailed | ||
otlp: | ||
endpoint: "http://pyroscope:4040" | ||
tls: | ||
insecure: true | ||
|
||
service: | ||
pipelines: | ||
traces: | ||
receivers: [otlp] | ||
processors: [batch] | ||
exporters: [debug] | ||
metrics: | ||
receivers: [otlp] | ||
processors: [batch] | ||
exporters: [debug] | ||
logs: | ||
receivers: [otlp] | ||
processors: [batch] | ||
exporters: [debug] | ||
profiles: | ||
receivers: [otlp] | ||
exporters: [otlp] | ||
|
||
telemetry: | ||
logs: | ||
level: debug |
54 changes: 54 additions & 0 deletions
54
examples/grafana-agent-auto-instrumentation/ebpf-otel/docker/docker-compose.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
version: '3' | ||
services: | ||
otel-collector: | ||
image: otel/opentelemetry-collector-contrib:latest | ||
command: ["--config=/etc/otel-collector-config.yaml", "--feature-gates=service.profilesSupport"] | ||
volumes: | ||
- ./config/otel-collector-config.yaml:/etc/otel-collector-config.yaml | ||
ports: | ||
- "4317:4317" | ||
- "4318:4318" | ||
networks: | ||
- otel-net | ||
depends_on: | ||
- pyroscope | ||
|
||
otel-ebpf-profiler: | ||
build: . | ||
hostname: ebpf-profiler | ||
privileged: true | ||
pid: "host" | ||
volumes: | ||
- /sys/kernel/debug:/sys/kernel/debug | ||
- /sys/fs/cgroup:/sys/fs/cgroup | ||
- /proc:/proc | ||
networks: | ||
- otel-net | ||
depends_on: | ||
- otel-collector | ||
|
||
pyroscope: | ||
image: grafana/pyroscope:latest | ||
command: ["-self-profiling.disable-push=true"] | ||
ports: | ||
- "4040:4040" | ||
networks: | ||
- otel-net | ||
|
||
grafana: | ||
image: grafana/grafana:latest | ||
environment: | ||
- GF_INSTALL_PLUGINS=grafana-pyroscope-app | ||
- GF_AUTH_ANONYMOUS_ENABLED=true | ||
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin | ||
- GF_AUTH_DISABLE_LOGIN_FORM=true | ||
volumes: | ||
- ./grafana-provisioning:/etc/grafana/provisioning | ||
ports: | ||
- "3000:3000" | ||
networks: | ||
- otel-net | ||
|
||
networks: | ||
otel-net: | ||
driver: bridge |
14 changes: 14 additions & 0 deletions
14
...gent-auto-instrumentation/ebpf-otel/docker/grafana-provisioning/datasources/pyroscope.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
--- | ||
apiVersion: 1 | ||
datasources: | ||
- uid: local-pyroscope | ||
type: grafana-pyroscope-datasource | ||
name: Pyroscope | ||
url: http://pyroscope:4040 | ||
jsonData: | ||
keepCookies: [pyroscope_git_session] | ||
# Uncomment these if using with Grafana Cloud | ||
# basicAuth: true | ||
# basicAuthUser: '123456' | ||
# secureJsonData: | ||
# basicAuthPassword: PASSWORD |
9 changes: 9 additions & 0 deletions
9
examples/grafana-agent-auto-instrumentation/ebpf-otel/kubernetes/Dockerfile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
FROM ubuntu:22.04 | ||
|
||
RUN apt-get update && \ | ||
apt-get install -y linux-headers-generic && \ | ||
rm -rf /var/lib/apt/lists/* | ||
|
||
COPY ebpf-profiler /usr/local/bin/ | ||
|
||
ENTRYPOINT ["/usr/local/bin/ebpf-profiler", "-collection-agent", "otel-collector-service:4317", "-no-kernel-version-check", "-disable-tls"] |
24 changes: 24 additions & 0 deletions
24
examples/grafana-agent-auto-instrumentation/ebpf-otel/kubernetes/cpu-stress.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: cpu-stress | ||
spec: | ||
replicas: 1 | ||
selector: | ||
matchLabels: | ||
app: cpu-stress | ||
template: | ||
metadata: | ||
labels: | ||
app: cpu-stress | ||
spec: | ||
containers: | ||
- name: stress | ||
image: polinux/stress | ||
command: ["stress"] | ||
args: ["--cpu", "2"] | ||
resources: | ||
limits: | ||
cpu: "2" | ||
requests: | ||
cpu: "1" |
74 changes: 74 additions & 0 deletions
74
examples/grafana-agent-auto-instrumentation/ebpf-otel/kubernetes/grafana.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: grafana | ||
spec: | ||
selector: | ||
matchLabels: | ||
app: grafana | ||
template: | ||
metadata: | ||
labels: | ||
app: grafana | ||
spec: | ||
containers: | ||
- name: grafana | ||
image: grafana/grafana:latest | ||
ports: | ||
- containerPort: 3000 | ||
env: | ||
- name: GF_INSTALL_PLUGINS | ||
value: grafana-pyroscope-app | ||
- name: GF_AUTH_ANONYMOUS_ENABLED | ||
value: "true" | ||
- name: GF_AUTH_ANONYMOUS_ORG_ROLE | ||
value: Admin | ||
- name: GF_AUTH_DISABLE_LOGIN_FORM | ||
value: "true" | ||
volumeMounts: | ||
- name: grafana-provisioning | ||
mountPath: /etc/grafana/provisioning | ||
volumes: | ||
- name: grafana-provisioning | ||
configMap: | ||
name: grafana-provisioning | ||
items: | ||
- key: datasources | ||
path: datasources/datasources.yaml | ||
- key: plugins | ||
path: plugins/plugins.yaml | ||
--- | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: grafana-service | ||
spec: | ||
selector: | ||
app: grafana | ||
ports: | ||
- protocol: TCP | ||
port: 3000 | ||
targetPort: 3000 | ||
--- | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
name: grafana-provisioning | ||
data: | ||
"datasources": | | ||
apiVersion: 1 | ||
datasources: | ||
- uid: local-pyroscope | ||
type: grafana-pyroscope-datasource | ||
name: Pyroscope | ||
url: http://pyroscope-service:4040 | ||
jsonData: | ||
keepCookies: [pyroscope_git_session] | ||
"plugins": | | ||
apiVersion: 1 | ||
apps: | ||
- type: grafana-pyroscope-app | ||
jsonData: | ||
backendUrl: http://pyroscope-service:4040 | ||
secureJsonData: |
Oops, something went wrong.