Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Gateway deployment arch #5452

Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ description:
Why and how to send signals to a single OTLP end-point and from there to
backends
weight: 3
cSpell:ignore: hostnames loadbalancer loadbalancing
# prettier-ignore
cSpell:ignore: filelogreceiver hostmetricsreceiver hostnames loadbalancer loadbalancing resourcedetection
---

The gateway collector deployment pattern consists of applications (or other
Expand Down Expand Up @@ -198,6 +199,42 @@ The load-balancing exporter emits metrics including
that you can use for health and performance monitoring of the OTLP endpoint
collector.

## Typical Gateway deployment architecture
dineshg13 marked this conversation as resolved.
Show resolved Hide resolved

A typical Gateway architecture involves running both Collector in Gateway mode
and Agent mode.
dineshg13 marked this conversation as resolved.
Show resolved Hide resolved

The following diagram shows a typical deployment architecture for a gateway
dineshg13 marked this conversation as resolved.
Show resolved Hide resolved
deployment. Here
dineshg13 marked this conversation as resolved.
Show resolved Hide resolved

- We use Agent Collectors (running on each host, similar to Kubernetes
daemonsets) to collect telemetry from services running on the host and host
telemetry, such as host metrics and scrap logs.
- We use Gateway Collectors to process OTLP data, such as filtering, sampling,
and exporting to backends etc.
dineshg13 marked this conversation as resolved.
Show resolved Hide resolved

![gateway](otel-gateway-arch.svg)
svrnm marked this conversation as resolved.
Show resolved Hide resolved

There are a few limitations in running the OpenTelemetry Collector in gateway
mode.

- There are few receivers in OpenTelemetry Collector that need to be unique per
host instance. Running multiple instances of these receivers will result in
duplicate data. It is recommended not to use them in the Gateway Collector but
only for the collector running in Agent mode. Examples include, but are not
limited to:

- [`hostmetricsreceiver`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver),
may result in duplicate host metrics.
- [`filelogreceiver`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver),
may result in duplicate logs.

- Using a
[`resourcedetection`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor)
processor in gateway mode is not recommended. This processor is used to detect
the resources of the host where the Collector is running. Use
`resourcedetection` processor in the pipeline of Agent collector deployment.
svrnm marked this conversation as resolved.
Show resolved Hide resolved

## Tradeoffs

Pros:
Expand Down
svrnm marked this conversation as resolved.
Show resolved Hide resolved
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.