[feat][misc] PIP-264: Copy OpenTelemetry resource attributes to Prometheus labels #23005
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PIP-264
Motivation
The OpenTelemetry collector can be configured to expose the metrics as a Prometheus server, simplifying the deployment and migration of the telemetry pipeline. Pulsar supports this feature out of the box.
There is a catch though: the Prometheus exporter did not copy the OpenTelemetry resource attributes over to corresponding Prometheus labels, so vital information was lost in this process. This limitation is described in open-telemetry/opentelemetry-java#6108, which has since been fixed. The present PR leverages this improvement, instructing the OpenTelemetry SDK to migrate all resource attributes over to Prometheus as labels.
Modifications
OpenTelemetryService
, configure any Prometheus exporters to copy all resource attributes as labels. Caveat: At the time this happens, the Prometheus server is already initialized and listening. Reconfiguring it on-the-go is not possible, so the server is closed and recreated with the desired configuration.Verifying this change
This change modified tests and can be verified as follows:
OpenTelemetrySanityTest#testOpenTelemetryMetricsPrometheusExport
to validate the labels are attached to the metrics exposed by the Prometheus exporter.Does this pull request potentially affect one of the following parts:
Documentation
doc
doc-required
[feat][doc] PIP-264: Add OpenTelemetry managed cursor metrics reference pulsar-site#929doc-not-needed
doc-complete
Matching PR in forked repository
PR in forked repository: dragosvictor#38