Skip to content

Commit

Permalink
Docs/Tests: documentation and e2e tests for Datadog tracing (envoypro…
Browse files Browse the repository at this point in the history
…xy#4480)

* Add e2e test for Datadog tracing

Signed-off-by: Hartigan <[email protected]>

* Add documentation for Datadog tracing support for proxy

Signed-off-by: Hartigan <[email protected]>

* Format test yaml files

Signed-off-by: Hartigan <[email protected]>

* Remove Grafana Alloy for Datadog tracing e2e test

Signed-off-by: Hartigan <[email protected]>

* Update generated files

Signed-off-by: Hartigan <[email protected]>

---------

Signed-off-by: Hartigan <[email protected]>
  • Loading branch information
Hartigan authored Oct 21, 2024
1 parent deddeb4 commit 33ac6ca
Show file tree
Hide file tree
Showing 10 changed files with 277 additions and 39 deletions.
6 changes: 3 additions & 3 deletions charts/gateway-addons-helm/Chart.lock
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ dependencies:
version: 1.3.1
- name: opentelemetry-collector
repository: https://open-telemetry.github.io/opentelemetry-helm-charts
version: 0.73.1
digest: sha256:4c16df8d7efc27aff566fa5dfd2eba6527adbf3fc8e94e7e3ccfc0cee7836f1c
generated: "2024-06-20T11:46:59.148579+08:00"
version: 0.108.0
digest: sha256:ea6663bb1358123b96b69d2c5b0b8c20650a43dc39b24c482f0560201fd2cc3a
generated: "2024-10-19T12:59:47.251089661+02:00"
2 changes: 1 addition & 1 deletion charts/gateway-addons-helm/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,5 +47,5 @@ dependencies:
condition: tempo.enabled
- name: opentelemetry-collector
repository: https://open-telemetry.github.io/opentelemetry-helm-charts
version: 0.73.1
version: 0.108.0
condition: opentelemetry-collector.enabled
14 changes: 9 additions & 5 deletions charts/gateway-addons-helm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ An Add-ons Helm chart for Envoy Gateway
| https://grafana.github.io/helm-charts | grafana | 8.0.0 |
| https://grafana.github.io/helm-charts | loki | 4.8.0 |
| https://grafana.github.io/helm-charts | tempo | 1.3.1 |
| https://open-telemetry.github.io/opentelemetry-helm-charts | opentelemetry-collector | 0.73.1 |
| https://open-telemetry.github.io/opentelemetry-helm-charts | opentelemetry-collector | 0.108.0 |
| https://prometheus-community.github.io/helm-charts | prometheus | 25.21.0 |

## Usage
Expand Down Expand Up @@ -103,7 +103,7 @@ To uninstall the chart:
| loki.singleBinary.replicas | int | `1` | |
| loki.test.enabled | bool | `false` | |
| loki.write.replicas | int | `0` | |
| opentelemetry-collector.config.exporters.logging.verbosity | string | `"detailed"` | |
| opentelemetry-collector.config.exporters.debug.verbosity | string | `"detailed"` | |
| opentelemetry-collector.config.exporters.loki.endpoint | string | `"http://loki.monitoring.svc:3100/loki/api/v1/push"` | |
| opentelemetry-collector.config.exporters.otlp.endpoint | string | `"tempo.monitoring.svc:4317"` | |
| opentelemetry-collector.config.exporters.otlp.tls.insecure | bool | `true` | |
Expand All @@ -112,6 +112,7 @@ To uninstall the chart:
| opentelemetry-collector.config.processors.attributes.actions[0].action | string | `"insert"` | |
| opentelemetry-collector.config.processors.attributes.actions[0].key | string | `"loki.attribute.labels"` | |
| opentelemetry-collector.config.processors.attributes.actions[0].value | string | `"k8s.pod.name, k8s.namespace.name"` | |
| opentelemetry-collector.config.receivers.datadog.endpoint | string | `"${env:MY_POD_IP}:8126"` | |
| opentelemetry-collector.config.receivers.otlp.protocols.grpc.endpoint | string | `"${env:MY_POD_IP}:4317"` | |
| opentelemetry-collector.config.receivers.otlp.protocols.http.endpoint | string | `"${env:MY_POD_IP}:4318"` | |
| opentelemetry-collector.config.receivers.zipkin.endpoint | string | `"${env:MY_POD_IP}:9411"` | |
Expand All @@ -120,12 +121,15 @@ To uninstall the chart:
| opentelemetry-collector.config.service.pipelines.logs.processors[0] | string | `"attributes"` | |
| opentelemetry-collector.config.service.pipelines.logs.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.metrics.exporters[0] | string | `"prometheus"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[0] | string | `"datadog"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[1] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.exporters[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[1] | string | `"zipkin"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[0] | string | `"datadog"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[1] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[2] | string | `"zipkin"` | |
| opentelemetry-collector.enabled | bool | `false` | |
| opentelemetry-collector.fullnameOverride | string | `"otel-collector"` | |
| opentelemetry-collector.image.repository | string | `"otel/opentelemetry-collector-contrib"` | |
| opentelemetry-collector.mode | string | `"deployment"` | |
| prometheus.alertmanager.enabled | bool | `false` | |
| prometheus.enabled | bool | `true` | |
Expand Down
8 changes: 7 additions & 1 deletion charts/gateway-addons-helm/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -181,11 +181,13 @@ opentelemetry-collector:
enabled: false
fullnameOverride: otel-collector
mode: deployment
image:
repository: "otel/opentelemetry-collector-contrib"
config:
exporters:
prometheus:
endpoint: 0.0.0.0:19001
logging:
debug:
verbosity: detailed
loki:
endpoint: "http://loki.monitoring.svc:3100/loki/api/v1/push"
Expand All @@ -207,6 +209,8 @@ opentelemetry-collector:
# Loki will convert this to k8s_pod_name label.
value: k8s.pod.name, k8s.namespace.name
receivers:
datadog:
endpoint: ${env:MY_POD_IP}:8126
zipkin:
endpoint: ${env:MY_POD_IP}:9411
otlp:
Expand All @@ -223,6 +227,7 @@ opentelemetry-collector:
exporters:
- prometheus
receivers:
- datadog
- otlp
logs:
exporters:
Expand All @@ -235,5 +240,6 @@ opentelemetry-collector:
exporters:
- otlp
receivers:
- datadog
- otlp
- zipkin
14 changes: 9 additions & 5 deletions site/content/en/latest/install/gateway-addons-helm-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ An Add-ons Helm chart for Envoy Gateway
| https://grafana.github.io/helm-charts | grafana | 8.0.0 |
| https://grafana.github.io/helm-charts | loki | 4.8.0 |
| https://grafana.github.io/helm-charts | tempo | 1.3.1 |
| https://open-telemetry.github.io/opentelemetry-helm-charts | opentelemetry-collector | 0.73.1 |
| https://open-telemetry.github.io/opentelemetry-helm-charts | opentelemetry-collector | 0.108.0 |
| https://prometheus-community.github.io/helm-charts | prometheus | 25.21.0 |

## Values
Expand Down Expand Up @@ -82,7 +82,7 @@ An Add-ons Helm chart for Envoy Gateway
| loki.singleBinary.replicas | int | `1` | |
| loki.test.enabled | bool | `false` | |
| loki.write.replicas | int | `0` | |
| opentelemetry-collector.config.exporters.logging.verbosity | string | `"detailed"` | |
| opentelemetry-collector.config.exporters.debug.verbosity | string | `"detailed"` | |
| opentelemetry-collector.config.exporters.loki.endpoint | string | `"http://loki.monitoring.svc:3100/loki/api/v1/push"` | |
| opentelemetry-collector.config.exporters.otlp.endpoint | string | `"tempo.monitoring.svc:4317"` | |
| opentelemetry-collector.config.exporters.otlp.tls.insecure | bool | `true` | |
Expand All @@ -91,6 +91,7 @@ An Add-ons Helm chart for Envoy Gateway
| opentelemetry-collector.config.processors.attributes.actions[0].action | string | `"insert"` | |
| opentelemetry-collector.config.processors.attributes.actions[0].key | string | `"loki.attribute.labels"` | |
| opentelemetry-collector.config.processors.attributes.actions[0].value | string | `"k8s.pod.name, k8s.namespace.name"` | |
| opentelemetry-collector.config.receivers.datadog.endpoint | string | `"${env:MY_POD_IP}:8126"` | |
| opentelemetry-collector.config.receivers.otlp.protocols.grpc.endpoint | string | `"${env:MY_POD_IP}:4317"` | |
| opentelemetry-collector.config.receivers.otlp.protocols.http.endpoint | string | `"${env:MY_POD_IP}:4318"` | |
| opentelemetry-collector.config.receivers.zipkin.endpoint | string | `"${env:MY_POD_IP}:9411"` | |
Expand All @@ -99,12 +100,15 @@ An Add-ons Helm chart for Envoy Gateway
| opentelemetry-collector.config.service.pipelines.logs.processors[0] | string | `"attributes"` | |
| opentelemetry-collector.config.service.pipelines.logs.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.metrics.exporters[0] | string | `"prometheus"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[0] | string | `"datadog"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[1] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.exporters[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[1] | string | `"zipkin"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[0] | string | `"datadog"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[1] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[2] | string | `"zipkin"` | |
| opentelemetry-collector.enabled | bool | `false` | |
| opentelemetry-collector.fullnameOverride | string | `"otel-collector"` | |
| opentelemetry-collector.image.repository | string | `"otel/opentelemetry-collector-contrib"` | |
| opentelemetry-collector.mode | string | `"deployment"` | |
| prometheus.alertmanager.enabled | bool | `false` | |
| prometheus.enabled | bool | `true` | |
Expand Down
62 changes: 61 additions & 1 deletion site/content/en/latest/tasks/observability/proxy-trace.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ TEMPO_IP=$(kubectl get svc tempo -n monitoring -o jsonpath='{.status.loadBalance

By default, Envoy Gateway doesn't send traces to any sink.
You can enable traces by setting the `telemetry.tracing` in the [EnvoyProxy][envoy-proxy-crd] CRD.
Currently, Envoy Gateway support OpenTelemetry and [Zipkin](../../api/extension_types#zipkintracingprovider) tracer.
Currently, Envoy Gateway support OpenTelemetry, [Zipkin](../../api/extension_types#zipkintracingprovider) and Datadog tracer.

### Tracing Provider

Expand Down Expand Up @@ -155,6 +155,66 @@ Verify zipkin traces from tempo:
curl -s "http://$TEMPO_IP:3100/api/search?tags=component%3Dproxy+provider%3Dzipkin" | jq .traces
```

{{% /tab %}}
{{% tab header="Datadog" %}}

```shell
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
parametersRef:
group: gateway.envoyproxy.io
kind: EnvoyProxy
name: datadog
namespace: envoy-gateway-system
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: datadog
namespace: envoy-gateway-system
spec:
telemetry:
tracing:
# sample 100% of requests
samplingRate: 100
provider:
backendRefs:
- name: datadog-agent
namespace: monitoring
port: 8126
type: Datadog
customTags:
# This is an example of using a literal as a tag value
provider:
type: Literal
literal:
value: "datadog"
"k8s.pod.name":
type: Environment
environment:
name: ENVOY_POD_NAME
defaultValue: "-"
"k8s.namespace.name":
type: Environment
environment:
name: ENVOY_GATEWAY_NAMESPACE
defaultValue: "envoy-gateway-system"
# This is an example of using a header value as a tag value
header1:
type: RequestHeader
requestHeader:
name: X-Header-1
defaultValue: "-"
EOF
```

Verify Datadog traces in [Datadog APM](https://docs.datadoghq.com/tracing/)

{{% /tab %}}
{{< /tabpane >}}

Expand Down
14 changes: 9 additions & 5 deletions site/content/zh/latest/install/gateway-addons-helm-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ An Add-ons Helm chart for Envoy Gateway
| https://grafana.github.io/helm-charts | grafana | 8.0.0 |
| https://grafana.github.io/helm-charts | loki | 4.8.0 |
| https://grafana.github.io/helm-charts | tempo | 1.3.1 |
| https://open-telemetry.github.io/opentelemetry-helm-charts | opentelemetry-collector | 0.73.1 |
| https://open-telemetry.github.io/opentelemetry-helm-charts | opentelemetry-collector | 0.108.0 |
| https://prometheus-community.github.io/helm-charts | prometheus | 25.21.0 |

## Values
Expand Down Expand Up @@ -82,7 +82,7 @@ An Add-ons Helm chart for Envoy Gateway
| loki.singleBinary.replicas | int | `1` | |
| loki.test.enabled | bool | `false` | |
| loki.write.replicas | int | `0` | |
| opentelemetry-collector.config.exporters.logging.verbosity | string | `"detailed"` | |
| opentelemetry-collector.config.exporters.debug.verbosity | string | `"detailed"` | |
| opentelemetry-collector.config.exporters.loki.endpoint | string | `"http://loki.monitoring.svc:3100/loki/api/v1/push"` | |
| opentelemetry-collector.config.exporters.otlp.endpoint | string | `"tempo.monitoring.svc:4317"` | |
| opentelemetry-collector.config.exporters.otlp.tls.insecure | bool | `true` | |
Expand All @@ -91,6 +91,7 @@ An Add-ons Helm chart for Envoy Gateway
| opentelemetry-collector.config.processors.attributes.actions[0].action | string | `"insert"` | |
| opentelemetry-collector.config.processors.attributes.actions[0].key | string | `"loki.attribute.labels"` | |
| opentelemetry-collector.config.processors.attributes.actions[0].value | string | `"k8s.pod.name, k8s.namespace.name"` | |
| opentelemetry-collector.config.receivers.datadog.endpoint | string | `"${env:MY_POD_IP}:8126"` | |
| opentelemetry-collector.config.receivers.otlp.protocols.grpc.endpoint | string | `"${env:MY_POD_IP}:4317"` | |
| opentelemetry-collector.config.receivers.otlp.protocols.http.endpoint | string | `"${env:MY_POD_IP}:4318"` | |
| opentelemetry-collector.config.receivers.zipkin.endpoint | string | `"${env:MY_POD_IP}:9411"` | |
Expand All @@ -99,12 +100,15 @@ An Add-ons Helm chart for Envoy Gateway
| opentelemetry-collector.config.service.pipelines.logs.processors[0] | string | `"attributes"` | |
| opentelemetry-collector.config.service.pipelines.logs.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.metrics.exporters[0] | string | `"prometheus"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[0] | string | `"datadog"` | |
| opentelemetry-collector.config.service.pipelines.metrics.receivers[1] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.exporters[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[0] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[1] | string | `"zipkin"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[0] | string | `"datadog"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[1] | string | `"otlp"` | |
| opentelemetry-collector.config.service.pipelines.traces.receivers[2] | string | `"zipkin"` | |
| opentelemetry-collector.enabled | bool | `false` | |
| opentelemetry-collector.fullnameOverride | string | `"otel-collector"` | |
| opentelemetry-collector.image.repository | string | `"otel/opentelemetry-collector-contrib"` | |
| opentelemetry-collector.mode | string | `"deployment"` | |
| prometheus.alertmanager.enabled | bool | `false` | |
| prometheus.enabled | bool | `true` | |
Expand Down
91 changes: 91 additions & 0 deletions test/e2e/testdata/tracing-datadog.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: eg-special-case-datadog
namespace: gateway-conformance-infra
spec:
gatewayClassName: "{GATEWAY_CLASS_NAME}"
listeners:
- name: http
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: All
infrastructure:
parametersRef:
group: gateway.envoyproxy.io
kind: EnvoyProxy
name: datadog-tracing
---
apiVersion: v1
kind: Service
metadata:
name: datadog-agent
namespace: monitoring
spec:
selector:
app.kubernetes.io/instance: eg-addons
app.kubernetes.io/name: opentelemetry-collector
component: standalone-collector
ports:
- protocol: TCP
port: 8126
targetPort: 8126
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
name: datadog-tracing
namespace: gateway-conformance-infra
spec:
logging:
level:
default: debug
telemetry:
tracing:
provider:
type: Datadog
backendRefs:
- name: datadog-agent
namespace: monitoring
port: 8126
customTags:
"provider":
type: Literal
literal:
value: "datadog"
"k8s.cluster.name":
type: Literal
literal:
value: "envoy-gateway"
"k8s.pod.name":
type: Environment
environment:
name: ENVOY_POD_NAME
defaultValue: "-"
"k8s.namespace.name":
type: Environment
environment:
name: ENVOY_GATEWAY_NAMESPACE
defaultValue: "envoy-gateway-system"
shutdown:
drainTimeout: 5s
minDrainDuration: 1s
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: tracing-datadog
namespace: gateway-conformance-infra
spec:
parentRefs:
- name: eg-special-case-datadog
rules:
- matches:
- path:
type: PathPrefix
value: /datadog
backendRefs:
- name: infra-backend-v2
port: 8080
Loading

0 comments on commit 33ac6ca

Please sign in to comment.