Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Gateway deployment arch #5452

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ description:
Why and how to send signals to a single OTLP end-point and from there to
backends
weight: 3
cSpell:ignore: hostnames loadbalancer loadbalancing
# prettier-ignore
cSpell:ignore: filelogreceiver hostmetricsreceiver hostnames loadbalancer loadbalancing resourcedetectionprocessor
---

The gateway collector deployment pattern consists of applications (or other
Expand Down Expand Up @@ -198,6 +199,38 @@ The load-balancing exporter emits metrics including
that you can use for health and performance monitoring of the OTLP endpoint
collector.

## Combined deployment of Collectors as agents and gateways

Often a deployment of multiple OpenTelemetry collectors involves running both
Collector as gateways and as [agents](/docs/collector/deployment/agent/).

The following diagram shows an architecture for such a combined deployment:

- We use the Collectors running in the agent deployment pattern (running on each
host, similar to Kubernetes daemonsets) to collect telemetry from services
running on the host and host telemetry, such as host metrics and scrap logs.
- We use Collectors running in the gateway deployment pattern to process data,
such as filtering, sampling, and exporting to backends etc.

![gateway](otel-gateway-arch.svg)

This combined deployment pattern is necessary, when you use components in your
Collector that either need to be unique per host or that consume information
that is only available on the same host as the application is running:

- Receivers like the
[`hostmetricsreceiver`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver)
or
[`filelogreceiver`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver)
need to be unique per host instance. Running multiple instances of these
receivers will result in duplicated data.

- Processors like the
[`resourcedetectionprocessor`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor)
are used to add information about the host, the collector and the application
are running on. Running them within a Collector on a remote machine will
result in incorrect data.

## Tradeoffs

Pros:
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.