Skip to content

Commit

Permalink
Merge branch 'main' into js-propagation-update-2
Browse files Browse the repository at this point in the history
  • Loading branch information
svrnm authored Jul 4, 2024
2 parents 9b46d7a + f6d3f3e commit 48f97a4
Show file tree
Hide file tree
Showing 260 changed files with 1,007 additions and 357 deletions.
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
[submodule "content-modules/opentelemetry-proto"]
path = content-modules/opentelemetry-proto
url = https://github.com/open-telemetry/opentelemetry-proto
otlp-pin = v1.3.1
otlp-pin = v1.3.2
[submodule "content-modules/semantic-conventions"]
path = content-modules/semantic-conventions
url = https://github.com/open-telemetry/semantic-conventions
Expand Down
149 changes: 149 additions & 0 deletions content/en/blog/2024/hardening-the-collector-one.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
---
title: 'Hardening the Collector Episode 1: A new default bind address'
linkTitle: A new default bind address for the Collector
date: 2024-07-02
author: '[Pablo Baeyens](https://github.com/mx-psi) (OpenTelemetry, Datadog)'
# prettier-ignore
cSpell:ignore: awsfirehose awsproxy awsxray Baeyens jaegerremotesampling loki OSTIF remotetap sapm signalfx skywalking splunk
issue: 4760
sig: Collector SIG
---

The OpenTelemetry Collector recently went through a security audit sponsored by
the [CNCF](https://www.cncf.io/), facilitated by [OSTIF](https://ostif.org/),
and performed by [7ASecurity](https://7asecurity.com/). As part of this process
we published a security advisory related to a
[DoS vulnerability](/blog/2024/cve-2024-36129/) that was
[fully addressed in v0.102.1](https://github.com/open-telemetry/opentelemetry-collector/releases/tag/v0.102.1).

The security audit also motivated us to think about ways to harden official
Collector builds and have a more secure default configuration. We are working on
adopting [several][releases-586] [best][core-10469] [practices][core-10470] that
were recommended in the audit to achieve this and we will be publishing a series
of blog posts to keep the community informed. While we expect the report to be
made public soon, we can already say that we are very satisfied with the
confirmation that the Collector has proven to be very secure, highlighting the
secure coding practices and processes we already have in place.

One of the changes we have been working on is changing the default bind address
for Collector servers, such as those exposed by receivers or extensions that
listen for incoming connections. Up to v0.103.0, the default behavior was to
listen on all network interfaces by using the
[unspecified address `0.0.0.0`](https://en.wikipedia.org/wiki/0.0.0.0) on server
addresses. While this is a convenient default for test cases and development
environments, it is
[not the recommended practice for production environments](https://cwe.mitre.org/data/definitions/1327.html),
since it can expose the Collector servers to unnecessary risks. Starting on
v0.104.0 the default bind address becomes `localhost` for all Collector servers.

It has been a long way to get here. We started discussing this in relation to
[CVE-2022-27664](https://github.com/advisories/GHSA-69cg-p879-7622) on [v0.63.0
(September 2022)][core-6151], when we added a warning and improved our
documentation. On [v0.94.0 (September 2023)][core-8510], we decided to add a
feature gate, `component.UseLocalHostAsDefaultHost` to allow users to opt-in to
the new behavior. Finally, this feature gate was enabled by default on [v0.104.0
(June 2024)][core-10352] motivated by the security audit and
[CVE-2024-36129](/blog/2024/cve-2024-36129/).

## What have we changed?

Starting on v0.104.0, the default bind address of all servers exposed by the
Collector are `localhost` instead of `0.0.0.0`. For example, the OTLP receiver
default endpoints for OTLP/gRPC and OTLP/HTTP are now `localhost:4317` and
`localhost:4318` respectively. The full list of components affected by this
change is:

- [`otlp` receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver#otlp-receiver)
- [`awsfirehose` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awsfirehosereceiver#aws-kinesis-data-firehose-receiver)
- [`awsxray` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/awsxrayreceiver#aws-x-ray-receiver)
- [`influxdb` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/influxdbreceiver#influxdb-receiver)
- [`jaeger` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/jaegerreceiver#jaeger-receiver)
- [`loki` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/lokireceiver#loki-receiver)
- [`opencensus` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/opencensusreceiver#opencensus-receiver)
- [`sapm` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/sapmreceiver#sapm-receiver)
- [`signalfx` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/signalfxreceiver#signalfx-receiver)
- [`skywalking` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/skywalkingreceiver#skywalking-receiver)
- [`splunk_hec` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/splunkhecreceiver#splunk-hec-receiver)
- [`zipkin` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/zipkinreceiver#zipkin-receiver)
- [`zookeeper` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/zookeeperreceiver#zookeeper-receiver)
- [`awsproxy` extension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/awsproxy#aws-proxy)
- [`health_check` extension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/healthcheckextension#health-check)
- [`jaegerremotesampling` extension](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/jaegerremotesampling#jaegers-remote-sampling-extension)
- [`remotetap` processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/remotetapprocessor#remote-tap-processor)

When in doubt, check the specific components' documentation to see the new
default values.

Starting on the [OpenTelemetry Collector Helm Chart][helm-chart] v0.47.1 and on
v0.87.0 of the OpenTelemetry Collector official Docker images we updated the
default configuration for all components to explicitly set the endpoints to an
explicit value.

## What does it mean to me?

If you are relying on the default configuration you may need to start explicitly
setting the endpoint on your Collector components. For example, if you are using
the following configuration with the OTLP receiver:

```yaml
receivers:
otlp:
protocols:
grpc:
```
You may now need to explicitly set the `otlp::protocols::grpc::endpoint`
[configuration setting](https://github.com/open-telemetry/opentelemetry-collector/blob/v0.103.0/receiver/otlpreceiver/config.md):

```yaml
receivers:
otlp:
protocols:
grpc:
endpoint: ${env:HOST_IP}:4317
```

where the `HOST_IP` environment variable is set to the bind address you want to
use (for example, `status.podIP` on Kubernetes).

Because of the changes in the Collector Helm Chart and Collector Docker images
you are not affected if using the default configuration on either of these.

## How can I prepare for this change?

Since v0.63.0 the Collector logs a warning related to this when you have an
endpoint using the 0.0.0.0 address. Before upgrading, you can check for this
warning and address it. From v0.94.0 to v0.103.0 you can also preview the impact
of this change by [enabling][feature-gate] the
`component.UseLocalHostAsDefaultHost` feature gate.

Addressing this change should be straightforward, however, due to the number of
components that are impacted, starting on v0.104.0 you can temporarily opt out
of this change by disabling the `component.UseLocalHostAsDefaultHost` feature
gate so you can work on addressing this at your own pace. This feature gate will
be marked as stable in a future Collector release, so we recommend addressing
this as soon as possible.

## What's next?

As we work on adopting the best practices recommended by the security audit, we
will be publishing more blog posts to keep the community informed. This will
include hardening the Collector binaries on macOS and further the default
behavior of Collector servers. Stay tuned!

[helm-chart]:
https://github.com/open-telemetry/opentelemetry-helm-charts?tab=readme-ov-file#opentelemetry-collector
[feature-gate]:
https://github.com/open-telemetry/opentelemetry-collector/tree/v0.103.0/featuregate#controlling-gates
[releases-586]:
https://github.com/open-telemetry/opentelemetry-collector-releases/issues/586
[core-6151]:
https://github.com/open-telemetry/opentelemetry-collector/issues/6151
[core-8510]:
https://github.com/open-telemetry/opentelemetry-collector/issues/8510
[core-10469]:
https://github.com/open-telemetry/opentelemetry-collector/issues/10469
[core-10470]:
https://github.com/open-telemetry/opentelemetry-collector/issues/10470
[core-10352]:
https://github.com/open-telemetry/opentelemetry-collector/pull/10352
2 changes: 1 addition & 1 deletion content/en/docs/collector/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Collector
description: Vendor-agnostic way to receive, process and export telemetry data.
aliases: [collector/about]
cascade:
vers: 0.103.0
vers: 0.104.0
weight: 270
---

Expand Down
2 changes: 1 addition & 1 deletion content/en/docs/collector/internal-telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,4 +332,4 @@ with the network or backend receiving the data.

You can monitor data ingress with the `otelcol_receiver_accepted_spans` and
`otelcol_receiver_accepted_metric_points` metrics and data egress with the
`otecol_exporter_sent_spans` and `otelcol_exporter_sent_metric_points` metrics.
`otelcol_exporter_sent_spans` and `otelcol_exporter_sent_metric_points` metrics.
83 changes: 44 additions & 39 deletions content/en/docs/concepts/signals/baggage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,46 +4,52 @@ weight: 4
description: Contextual information that is passed between signals.
---

In OpenTelemetry, Baggage is contextual information that’s passed between spans.
It's a key-value store that resides alongside span context in a trace, making
values available to any span created within that trace.
In OpenTelemetry, Baggage is contextual information that resides next to
context. Baggage is a key-value store, which means it lets you
[propagate](/docs/concepts/context-propagation/#propagation) any data you like
alongside [context](/docs/concepts/context-propagation/#context).

For example, imagine you want to have a `ClientId` attribute on every span in
your trace, which involves multiple services; however, `ClientId` is only
available in one specific service. To accomplish your goal, you can use
OpenTelemetry Baggage to propagate this value across your system.
Baggage means you can pass data across services and processes, making it
available to add to [traces](/docs/concepts/signals/traces/),
[metrics](/docs/concepts/signals/metrics/), or
[logs](/docs/concepts/signals/logs/) in those services.

OpenTelemetry uses
[Context Propagation](/docs/concepts/signals/traces/#context-propagation) to
pass Baggage around, and each of the different library implementations has
propagators that parse and make that Baggage available without you needing to
explicitly implement it.
## Example

![OTel Baggage](/img/otel-baggage.svg)
Baggage is often used in tracing to propagate additional data across services.

For example, imagine you have a `clientId` at the start of a request, but you'd
like for that ID to be available on all spans in a trace, some metrics in
another service, and some logs along the way. Because the trace may span
multiple services, you need some way to propagate that data without copying the
`clientId` across many places in your codebase.

## Why does OTel Baggage exist?
By using
[Context Propagation](/docs/concepts/signals/traces/#context-propagation) to
pass baggage across these services, the `clientId` is available to add to any
additional spans, metrics, or logs. Additionally, instrumentations automatically
propagate baggage for you.

Baggage provides a uniform way to store and propagate information across a trace
and other signals. For example, you may want to attach information from your
application to a span and retrieve that information much later and use it later
on with another span. However, spans in OpenTelemetry are immutable once
created, and can be exported before you need information on them later on.
Baggage allows you to work around this problem by providing a place to store and
retrieve information.
![OTel Baggage](/img/otel-baggage.svg)

## What should OTel Baggage be used for?

Common use cases include information that’s only accessible further up a stack.
This can include things like Account Identification, User IDs, Product IDs, and
origin IPs, for example. Passing these down your stack allows you to then add
them to your Spans in downstream services to make it easier to filter when
you’re searching in your Observability backend.
Baggage is best used to include information typically available only at the
start of a request further downstream. This can include things like Account
Identification, User IDs, Product IDs, and origin IPs, for example.

Propagating this information using baggage allows for deeper analysis of
telemetry in a backend. For example, if you include information like a User ID
on a span that tracks a database call, you can much more easily answer questions
like "which users are experiencing the slowest database calls?" You can also log
information about a downstream operation and include that same User ID in the
log data.

![OTel Baggage](/img/otel-baggage-2.svg)

## Baggage security considerations

Sensitive Baggage items could be shared with unintended resources, like
Sensitive Baggage items can be shared with unintended resources, like
third-party APIs. This is because automatic instrumentation includes Baggage in
most of your service’s network requests. Specifically, Baggage and other parts
of trace context are sent in HTTP headers, making it visible to anyone
Expand All @@ -52,22 +58,21 @@ then this risk may not apply, but keep in mind that downstream services could
propagate Baggage outside your network.

Also, there are no built-in integrity checks to ensure that Baggage items are
yours, so exercise caution when retrieving them.
yours, so exercise caution when reading them.

## Baggage is not the same as Span attributes
## Baggage is not the same as attributes

One important thing to note about Baggage is that it is not a subset of the
[Span Attributes](/docs/concepts/signals/traces/#attributes). When you add
something as Baggage, it does not automatically end up on the Attributes of the
child system’s spans. You must explicitly take something out of Baggage and
append it as Attributes.
An important thing to note about baggage is that it is a separate key-value
store and is unassociated with attributes on spans, metrics, or logs without
explicitly adding them.

For example, in .NET you might do this:
To add baggage entries to attributes, you need to explicitly read the data from
baggage and add it as attributes to your spans, metrics, or logs.

```csharp
var accountId = Baggage.GetBaggage("AccountId");
Activity.Current?.SetTag("AccountId", accountId);
```
Because a common use cases for Baggage is to add data to
[Span Attributes](/docs/concepts/signals/traces/#attributes) across a whole
trace, several languages have Baggage Span Processors that add data from baggage
as attributes on span creation.

> For more information, see the [baggage specification][].
Expand Down
4 changes: 2 additions & 2 deletions content/en/docs/contributing/blog.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,13 +79,13 @@ Follow these steps to create a post from the template:
1. Run the following command from the repository root:

```sh
npx hugo new content/en/blog/2023/short-name-for-post.md
npx hugo new content/en/blog/2024/short-name-for-post.md
```

If your post has images or other assets, run the following command:

```sh
npx hugo new content/en/blog/2023/short-name-for-post/index.md
npx hugo new content/en/blog/2024/short-name-for-post/index.md
```

1. Edit the Markdown file at the path you provided in the previous command. The
Expand Down
Loading

0 comments on commit 48f97a4

Please sign in to comment.