Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add section on how to build an otelcol dist #1234

Conversation

jpkrohling
Copy link
Member

Signed-off-by: Juraci Paixão Kröhling [email protected]

Description: Added readme section with links to resources on how to create custom OpenTelemetry Collector distributions.

Link to tracking Issue: open-telemetry/opentelemetry-collector#213

Testing: n/a

Documentation: n/a

@jpkrohling jpkrohling requested a review from a team October 9, 2020 14:04
README.md Outdated Show resolved Hide resolved
@bogdandrutu bogdandrutu changed the title Added section on how to build an otelcol dist Add section on how to build an otelcol dist Oct 12, 2020
Signed-off-by: Juraci Paixão Kröhling <[email protected]>
Signed-off-by: Juraci Paixão Kröhling <[email protected]>
@jpkrohling jpkrohling force-pushed the jpkrohling/add-link-to-blog-post-on-building-custom-otelcol branch from 73f9801 to acf4d51 Compare October 12, 2020 15:31
## Creating your own OpenTelemetry Collector distribution

A guide on how to create your own distribution is available in this blog post:
["Building your own OpenTelemetry Collector distribution"](https://medium.com/p/42337e994b63)
Copy link
Contributor

@owais owais Oct 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be wrong but isn't medium content sometimes locked behind a paywall now? Not sure if a freely available project both as in beer and speech should rely on extended documentation that might not be accessible to everyone.

Nothing against medium or content they host. Just not sure if OpenTelemetry as a product should direct users to content/documentation/guides that they might not be able to read.

Great article BTW @jpkrohling

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see this is the official OpenTelemetry blog on medium. I suppose this decision has already been taken then by the maintainers and we are fine with publishin there? Please disregard if that's the case.

Copy link
Member Author

@jpkrohling jpkrohling Oct 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sometimes locked behind a paywall now

Each author/publication can choose to make money on Medium and have a paywall in front of individual posts, or to allow all users to have free access to the content, without any paywalls. I always publish articles without opting-in for the paywall, and I believe the OpenTelemetry publication also doesn't.

README.md Show resolved Hide resolved
@bogdandrutu bogdandrutu merged commit a987792 into open-telemetry:master Oct 14, 2020
kisieland referenced this pull request in kisieland/opentelemetry-collector-contrib Mar 16, 2021
ljmsc referenced this pull request in ljmsc/opentelemetry-collector-contrib Feb 21, 2022
codeboten pushed a commit that referenced this pull request Nov 23, 2022
… add 'messaging.url' span attribute (#1234)

* boto3sqs: Fix various issues

* do not use 'otel' prefix for propagation keys to make propagation
  compatible with other SQS instrumentations like Node.Js
  Inject propergator.fields keys into the MessageAttributeNames argument
  for 'receive_message' calls to retreive the corresponding message attributes
* add 'messaging.url' span attribute to SQS spans
* add boto3sqs instrumentation to tox.ini to run tests in CI
* add some basic unit tests

* changelog

* fix linting issues

* unset instrumented flag on uninstrument
mx-psi added a commit that referenced this pull request Oct 4, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Removes `cmd/oteltestbedcol` from list since it has no Makefile

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes #35606
jriguera pushed a commit to springernature/opentelemetry-collector-contrib that referenced this pull request Oct 4, 2024
)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Removes `cmd/oteltestbedcol` from list since it has no Makefile

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#35606
Eromosele-SM pushed a commit to sematext/opentelemetry-collector-contrib that referenced this pull request Oct 9, 2024
)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Removes `cmd/oteltestbedcol` from list since it has no Makefile

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#35606
evan-bradley pushed a commit that referenced this pull request Oct 10, 2024
…atus (#35648)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Refactors the `extension/healthcheckv2extension/internal/status` into
`pkg/status`.

This exposes the aggregator to be used by other extensions to gather
component status information using the `extension.StatusWatcher`.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

Closes
#34692

<!--Describe what testing was performed and which tests were added.-->
#### Testing

Being it was a refactor and all the same tests provided coverage, no
additional testing was added.

<!--Describe the documentation added.-->
#### Documentation

Added a `README.md` to the `pkg/status` to provide information on where
this package can be used.

---------

Co-authored-by: Matthew Wear <[email protected]>
andrzej-stencel pushed a commit that referenced this pull request Oct 15, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Update OTel mode to implementation to serialize log body into body.*
fields

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
andrzej-stencel pushed a commit that referenced this pull request Oct 15, 2024
…rning error to avoid upstream retries (#35740)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Log metrics validation error instead of returning to avoid upstream
retries

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
andrzej-stencel pushed a commit that referenced this pull request Oct 16, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue #35741
Fixes 
Wrong comparison func implement in translator/prometheusremotewrite.

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->

Signed-off-by: Juraj Michalek <[email protected]>
andrzej-stencel pushed a commit that referenced this pull request Oct 16, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Follow up from #35703.

Draft starting the work on adding support for remote write 2.0 in the
translation package.
Adding support for translating gauges.

This is first iteration and to keep the PR small
* we don't handle duplicate metrics
* only support gauges
* don't handle other labels than metric name
* don't handle exemplars
*  don't handle metadata

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue #33661
Fixes

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: Juraj Michalek <[email protected]>
Co-authored-by: Arthur Silva Sens <[email protected]>
Co-authored-by: David Ashpole <[email protected]>
djaglowski pushed a commit that referenced this pull request Oct 16, 2024
#### Description
Follow [the
contract](https://github.com/open-telemetry/opentelemetry-collector/blob/df3c9e38a80ccc3b14705462be2e2e51c628a3b3/receiver/doc.go#L10)
for cloudflare receiver
<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Related
#5909

<!--Describe what testing was performed and which tests were added.-->
#### Testing

Added
MovieStoreGuy pushed a commit that referenced this pull request Oct 17, 2024
#### Description
Add a new receiver to handle systemd service state reporting as metrics.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

#33532
andrzej-stencel pushed a commit that referenced this pull request Oct 17, 2024
… prefix conflict in OTel mapping mode (#35651)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Metric names should be flattened and exported as is, even when one
metric name is a prefix of another. Same for attributes for all logs,
metrics and traces.


<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
djaglowski pushed a commit that referenced this pull request Oct 17, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Version of MySQL older than 8.0.22 use the `show slave status` vs `show
replica status` view to retrieve replication metrics. This allows for
both the newer & older views to work and retrieve data for the two
actual replica metrics available (`mysql.replica.time_behind_source` &
`mysql.replica.sql_delay`).

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes #35217

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Manual testing against MySQL versions 8.0, 8.0.37, & MariaDB Ver 15.1
Distrib 10.5.26-MariaDB

<!--Describe the documentation added.-->
#### Documentation
None needed

<!--Please delete paragraphs that you did not use before submitting.-->
MovieStoreGuy pushed a commit that referenced this pull request Oct 18, 2024
#### Description
Adds initial implementation of ntpreceiver.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
#34375
andrzej-stencel pushed a commit that referenced this pull request Oct 18, 2024
…only (#35839)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Make OTel mapping mode use RequireDataStream in docappender, which means
it will only send to data streams. This prevents auto creating regular
indices in OTel mapping mode due to a race condition in Elasticsearch
where otel-data index templates are not ready.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
andrzej-stencel pushed a commit that referenced this pull request Oct 21, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Enable gzip compression by default, at hardcoded level BestSpeed. To
disable compression, set `compression` to `none`.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->
djaglowski pushed a commit that referenced this pull request Oct 22, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Fixes an issue where the Kafka receiver would block on shutdown.

There was an earlier fix for this issue
[here](#32720).
This does solve the issue, but it was only applied to the traces
receiver, not the logs or metrics receiver.

The issue is this go routine in the `Start()` functions for logs and
metrics:
```go
go func() {
        if err := c.consumeLoop(ctx, metricsConsumerGroup); err != nil {
		componentstatus.ReportStatus(host, componentstatus.NewFatalErrorEvent(err))
	}
}()
```

The `consumeLoop()` function returns a `context.Canceled` error when
`Shutdown()` is called, which is expected. However
`componentstatus.ReportStatus()` blocks while attempting to report this
error. The reason/bug for this can be found
[here](open-telemetry/opentelemetry-collector#9824).

The previously mentioned PR fixed this for the traces receiver by
checking if the error returned by `consumeLoop()` is `context.Canceled`:
```go
go func() {
	if err := c.consumeLoop(ctx, consumerGroup); !errors.Is(err, context.Canceled) {
		componentstatus.ReportStatus(host, componentstatus.NewFatalErrorEvent(err))
	}
}()
```

Additionally, this is `consumeLoop()` for the traces receiver, with the
logs and metrics versions being identical:
```go
func (c *kafkaTracesConsumer) consumeLoop(ctx context.Context, handler sarama.ConsumerGroupHandler) error {
	for {
		// `Consume` should be called inside an infinite loop, when a
		// server-side rebalance happens, the consumer session will need to be
		// recreated to get the new claims
		if err := c.consumerGroup.Consume(ctx, c.topics, handler); err != nil {
			c.settings.Logger.Error("Error from consumer", zap.Error(err))
		}
		// check if context was cancelled, signaling that the consumer should stop
		if ctx.Err() != nil {
			c.settings.Logger.Info("Consumer stopped", zap.Error(ctx.Err()))
			return ctx.Err()
		}
	}
}
```

This does fix the issue, however the only error that can be returned by
`consumeLoop()` is a canceled context. When we create the context and
cancel function, we use `context.Background()`:
```go
ctx, cancel := context.WithCancel(context.Background())
```
This context is only used by `consumeLoop()` and the cancel function is
only called in `Shutdown()`.

Because `consumeLoop()` can only return a `context.Canceled` error, this
PR removes this unused code for the logs, metrics, and traces receivers.
Instead, `consumeLoop()` still logs the `context.Canceled` error but it
does not return any error and the go routine simply just calls
`consumeLoop()`.

Additional motivation for removing the call to
`componentstatus.ReportStatus()` is the underlying function called by
it, `componentstatus.Report()` says it does not need to be called during
`Shutdown()` or `Start()` as the service already does so for the given
component, [comment
here](https://github.com/open-telemetry/opentelemetry-collector/blob/main/component/componentstatus/status.go#L21-L25).
Even if there wasn't a bug causing this call to block, the component
still shouldn't call it since it would only be called during
`Shutdown()`.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes #30789

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Tested in a build of the collector with these changes scraping logs from
a Kafka instance. When the collector is stopped and `Shutdown()` gets
called, the receiver did not block and the collector stopped gracefully
as expected.
TylerHelmuth pushed a commit that referenced this pull request Oct 23, 2024
…llback (#35769)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Out of the discussions at
#35758
I realized that `ProcessWithCallback` does not handle the error returned
by the callback function. Since the error of the parse function is
handled I think we should handle this of the callback too.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: ChrsMark <[email protected]>
TylerHelmuth pushed a commit that referenced this pull request Oct 23, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Adding full support for handling labels.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue #33661
Fixes

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: Juraj Michalek <[email protected]>
Co-authored-by: Arthur Silva Sens <[email protected]>
Co-authored-by: David Ashpole <[email protected]>
djaglowski pushed a commit that referenced this pull request Oct 24, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

This PR introduces a benchmark test for the receiver adapter which tests
the performance of handling the log entries sent to the `LogEmitter`,
which are then received and converted by the receiver adapter. The
changes have been made based on the comment linked here:
#35669 (comment)

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Part of #35453

<!--Describe what testing was performed and which tests were added.-->
#### Testing

Added additional benchmark tests

---------

Signed-off-by: Florian Bacher <[email protected]>
andrzej-stencel pushed a commit that referenced this pull request Oct 25, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

This PR is to add `otlp_v1` into the list of valid `record_type` input.
Otherwise when `record_type` sets to `otlp_v1`, we get error:
```
2024-10-23T16:53:09.823Z	info	[email protected]/service.go:135	Setting up own telemetry...
2024-10-23T16:53:09.824Z	info	telemetry/metrics.go:70	Serving metrics	{"address": "localhost:8888", "metrics level": "Normal"}
Error: failed to build pipelines: failed to create "awsfirehose" receiver for data type "metrics": unrecognized record type
```
<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->

---------

Co-authored-by: Matthew Wear <[email protected]>
djaglowski pushed a commit that referenced this pull request Oct 28, 2024
#35834)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Fixed issue in which Operators with DropOnErrorQuiet are not dropping
log entries on error.

Note: This issue was introduced by a bug fix meant to ensure Silent
Operators are not logging errors (#35010). With this fix,
this side effect bug has been resolved.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes 35010

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Added UT
MovieStoreGuy pushed a commit that referenced this pull request Oct 29, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Add new uptime scraper to emit the `system.uptime` metric

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes #31627 

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Unit test added. Also tested locally (Linux VM) with following config:
```yaml
receivers:
  hostmetrics:
    scrapers:
      uptime:
exporters:
  debug:
    verbosity: detailed
service:
  pipelines:
    metrics:
      receivers:
        - hostmetrics
      exporters:
        - debug
```
<details><summary>Details</summary>
<p>

```console
$ make genotelcontribcol
[...]

$ make RUN_CONFIG=config-uptime.yaml run
2024-10-23T13:24:48.195+0200    info    [email protected]/service.go:135 Setting up own telemetry...
2024-10-23T13:24:48.195+0200    info    telemetry/metrics.go:70 Serving metrics {"address": "localhost:8888", "metrics level": "Normal"}
2024-10-23T13:24:48.195+0200    info    builders/builders.go:26 Development component. May change in the future.        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2024-10-23T13:24:48.199+0200    info    [email protected]/service.go:207 Starting otelcontribcol...      {"Version": "0.112.0-dev", "NumCPU": 7}
2024-10-23T13:24:48.199+0200    info    extensions/extensions.go:39     Starting extensions...
2024-10-23T13:24:48.199+0200    info    [email protected]/service.go:230 Everything is ready. Begin running and processing data.
2024-10-23T13:24:49.202+0200    info    Metrics {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2024-10-23T13:24:49.202+0200    info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/uptimescraper 0.112.0-dev
Metric #0
Descriptor:
     -> Name: system.uptime
     -> Description: The time the system has been running
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2024-10-22 18:23:22 +0000 UTC
Timestamp: 2024-10-23 11:24:49.202512199 +0000 UTC
Value: 61287.000000
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2024-10-23T13:25:49.206+0200    info    Metrics {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2024-10-23T13:25:49.206+0200    info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/uptimescraper 0.112.0-dev
Metric #0
Descriptor:
     -> Name: system.uptime
     -> Description: The time the system has been running
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2024-10-22 18:23:22 +0000 UTC
Timestamp: 2024-10-23 11:25:49.206657828 +0000 UTC
Value: 61347.000000
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2024-10-23T13:26:49.205+0200    info    Metrics {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2024-10-23T13:26:49.205+0200    info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/uptimescraper 0.112.0-dev
Metric #0
Descriptor:
     -> Name: system.uptime
     -> Description: The time the system has been running
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2024-10-22 18:23:22 +0000 UTC
Timestamp: 2024-10-23 11:26:49.2051249 +0000 UTC
Value: 61407.000000
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2024-10-23T13:27:49.210+0200    info    Metrics {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2024-10-23T13:27:49.210+0200    info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/uptimescraper 0.112.0-dev
Metric #0
Descriptor:
     -> Name: system.uptime
     -> Description: The time the system has been running
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2024-10-22 18:23:22 +0000 UTC
Timestamp: 2024-10-23 11:27:49.21007371 +0000 UTC
Value: 61467.000000
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
^C
2024-10-23T13:27:54.204+0200  info    [email protected]/collector.go:328       Received signal from OS {"signal": "interrupt"}
2024-10-23T13:27:54.204+0200    info    [email protected]/service.go:266 Starting shutdown...
2024-10-23T13:27:54.204+0200    info    extensions/extensions.go:66     Stopping extensions...
2024-10-23T13:27:54.204+0200    info    [email protected]/service.go:280 Shutdown complete.
```

</p>
</details> 


<!--Describe the documentation added.-->
#### Documentation
Updated readme & generated scraper docs

<!--Please delete paragraphs that you did not use before submitting.-->
MovieStoreGuy pushed a commit that referenced this pull request Oct 29, 2024
…SON status: ok (#36029)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Changing the `/api/v1/check_run` response for service checks to be on
par with the datadog api spec as per
https://docs.datadoghq.com/api/latest/service-checks/

This is the same implementation as the PR for the series endpoints
#35744

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

Fixes
#36027

<!--Describe what testing was performed and which tests were added.-->
#### Testing

Unit tests updated and are passing

<!--Describe the documentation added.-->
#### Documentation

Changelog updated
jpbarto pushed a commit to jpbarto/opentelemetry-collector-contrib that referenced this pull request Oct 29, 2024
open-telemetry#35834)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Fixed issue in which Operators with DropOnErrorQuiet are not dropping
log entries on error.

Note: This issue was introduced by a bug fix meant to ensure Silent
Operators are not logging errors (open-telemetry#35010). With this fix,
this side effect bug has been resolved.

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes 35010

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Added UT
jpbarto pushed a commit to jpbarto/opentelemetry-collector-contrib that referenced this pull request Oct 29, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Add new uptime scraper to emit the `system.uptime` metric

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#31627 

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Unit test added. Also tested locally (Linux VM) with following config:
```yaml
receivers:
  hostmetrics:
    scrapers:
      uptime:
exporters:
  debug:
    verbosity: detailed
service:
  pipelines:
    metrics:
      receivers:
        - hostmetrics
      exporters:
        - debug
```
<details><summary>Details</summary>
<p>

```console
$ make genotelcontribcol
[...]

$ make RUN_CONFIG=config-uptime.yaml run
2024-10-23T13:24:48.195+0200    info    [email protected]/service.go:135 Setting up own telemetry...
2024-10-23T13:24:48.195+0200    info    telemetry/metrics.go:70 Serving metrics {"address": "localhost:8888", "metrics level": "Normal"}
2024-10-23T13:24:48.195+0200    info    builders/builders.go:26 Development component. May change in the future.        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2024-10-23T13:24:48.199+0200    info    [email protected]/service.go:207 Starting otelcontribcol...      {"Version": "0.112.0-dev", "NumCPU": 7}
2024-10-23T13:24:48.199+0200    info    extensions/extensions.go:39     Starting extensions...
2024-10-23T13:24:48.199+0200    info    [email protected]/service.go:230 Everything is ready. Begin running and processing data.
2024-10-23T13:24:49.202+0200    info    Metrics {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2024-10-23T13:24:49.202+0200    info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/uptimescraper 0.112.0-dev
Metric #0
Descriptor:
     -> Name: system.uptime
     -> Description: The time the system has been running
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2024-10-22 18:23:22 +0000 UTC
Timestamp: 2024-10-23 11:24:49.202512199 +0000 UTC
Value: 61287.000000
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2024-10-23T13:25:49.206+0200    info    Metrics {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2024-10-23T13:25:49.206+0200    info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/uptimescraper 0.112.0-dev
Metric #0
Descriptor:
     -> Name: system.uptime
     -> Description: The time the system has been running
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2024-10-22 18:23:22 +0000 UTC
Timestamp: 2024-10-23 11:25:49.206657828 +0000 UTC
Value: 61347.000000
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2024-10-23T13:26:49.205+0200    info    Metrics {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2024-10-23T13:26:49.205+0200    info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/uptimescraper 0.112.0-dev
Metric #0
Descriptor:
     -> Name: system.uptime
     -> Description: The time the system has been running
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2024-10-22 18:23:22 +0000 UTC
Timestamp: 2024-10-23 11:26:49.2051249 +0000 UTC
Value: 61407.000000
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
2024-10-23T13:27:49.210+0200    info    Metrics {"kind": "exporter", "data_type": "metrics", "name": "debug", "resource metrics": 1, "metrics": 1, "data points": 1}
2024-10-23T13:27:49.210+0200    info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.9.0
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/uptimescraper 0.112.0-dev
Metric #0
Descriptor:
     -> Name: system.uptime
     -> Description: The time the system has been running
     -> Unit: s
     -> DataType: Sum
     -> IsMonotonic: false
     -> AggregationTemporality: Cumulative
NumberDataPoints #0
StartTimestamp: 2024-10-22 18:23:22 +0000 UTC
Timestamp: 2024-10-23 11:27:49.21007371 +0000 UTC
Value: 61467.000000
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
^C
2024-10-23T13:27:54.204+0200  info    [email protected]/collector.go:328       Received signal from OS {"signal": "interrupt"}
2024-10-23T13:27:54.204+0200    info    [email protected]/service.go:266 Starting shutdown...
2024-10-23T13:27:54.204+0200    info    extensions/extensions.go:66     Stopping extensions...
2024-10-23T13:27:54.204+0200    info    [email protected]/service.go:280 Shutdown complete.
```

</p>
</details> 


<!--Describe the documentation added.-->
#### Documentation
Updated readme & generated scraper docs

<!--Please delete paragraphs that you did not use before submitting.-->
jpbarto pushed a commit to jpbarto/opentelemetry-collector-contrib that referenced this pull request Oct 29, 2024
…SON status: ok (open-telemetry#36029)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

Changing the `/api/v1/check_run` response for service checks to be on
par with the datadog api spec as per
https://docs.datadoghq.com/api/latest/service-checks/

This is the same implementation as the PR for the series endpoints
open-telemetry#35744

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

Fixes
open-telemetry#36027

<!--Describe what testing was performed and which tests were added.-->
#### Testing

Unit tests updated and are passing

<!--Describe the documentation added.-->
#### Documentation

Changelog updated
mx-psi pushed a commit that referenced this pull request Oct 29, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
- This PR changes stability to `alpha`
- Updates README.md for more clarity and detailed config


<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
[#32669]

<!--Describe what testing was performed and which tests were added.-->
#### Testing
No additional testing needed

<!--Describe the documentation added.-->
#### Documentation
Adds clarification of config, additional detail to detailed config, and
notes about when and how to use which telemetry type.

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Co-authored-by: jeremyh <[email protected]>
mx-psi pushed a commit that referenced this pull request Oct 30, 2024
…ce detector (#35737)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

This PR fixes the issue described at
#34210.
Each detector's docs should list the resource attributes as they are
defined by the `metadata.yml` files.

This PR links each detector to its `documentation.md` as a reference for
the respective Resource Atrributes.



<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes
#34210

<!--Describe what testing was performed and which tests were added.-->
#### Testing ~

<!--Describe the documentation added.-->
#### Documentation
Added
<!--Please delete paragraphs that you did not use before submitting.-->

Signed-off-by: ChrsMark <[email protected]>
andrzej-stencel pushed a commit that referenced this pull request Oct 30, 2024
…e system is mounted (#36000)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Do not set the default value of HOST_PROC_MOUNTINFO to respect root_path

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes #35990
djaglowski pushed a commit that referenced this pull request Oct 31, 2024
)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Allows the Supervisor's OpAmp server port to be configurable. This is
useful for restricted environments deploying the supervisor.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes #36001 

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updated tests and added an e2e test

<!--Describe the documentation added.-->
#### Documentation
Spec was updated accordingly. Also added documentation for previously
added `health_check_port` that wasn't present in spec.

<!--Please delete paragraphs that you did not use before submitting.-->
TylerHelmuth pushed a commit that referenced this pull request Oct 31, 2024
… paths context (#35716)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

This PR is part of
#29017,
and adds the `ottl.Parser[K].AppendStatementPathsContext` function,
allowing components to rewrite statements appending missing `ottl.path`
context names.

For examples, the following context-less statement:

```
set(value, 1) where name == attributes["foo.name"]
```

Would be rewritten using the `span` context as:

```

set(span.value, 1) where span.name == span.attributes["foo.name"]
```

**Why do we need to rewrite statements?**

This utility will be used during the transition from structured OTTL
statements to flat statements.
Components such as the `transformprocessor` will leverage it to support
both configuration styles, without forcing
users to adapt/rewrite their existing config files. 

Once the component turns on the `ottl.Parser[K]` path's context
validation, new configuration style usages will be validated, requiring
all paths to have a context prefix, and old configuration styles will
automatically rewrite the statements using this function.

For more details, please have a look at the complete
[draft](#35050)
implementation.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

#29017

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Unit tests

<!--Describe the documentation added.-->
#### Documentation
No changes

<!--Please delete paragraphs that you did not use before submitting.-->
bogdandrutu pushed a commit that referenced this pull request Nov 3, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Fix memory leak by changing how we run obsreports for metrics and logs.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes
#35294
ArthurSens pushed a commit to ArthurSens/opentelemetry-collector-contrib that referenced this pull request Nov 4, 2024
…n-telemetry#36002)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Allows the Supervisor's OpAmp server port to be configurable. This is
useful for restricted environments deploying the supervisor.

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#36001 

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Updated tests and added an e2e test

<!--Describe the documentation added.-->
#### Documentation
Spec was updated accordingly. Also added documentation for previously
added `health_check_port` that wasn't present in spec.

<!--Please delete paragraphs that you did not use before submitting.-->
ArthurSens pushed a commit to ArthurSens/opentelemetry-collector-contrib that referenced this pull request Nov 4, 2024
… paths context (open-telemetry#35716)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

This PR is part of
open-telemetry#29017,
and adds the `ottl.Parser[K].AppendStatementPathsContext` function,
allowing components to rewrite statements appending missing `ottl.path`
context names.

For examples, the following context-less statement:

```
set(value, 1) where name == attributes["foo.name"]
```

Would be rewritten using the `span` context as:

```

set(span.value, 1) where span.name == span.attributes["foo.name"]
```

**Why do we need to rewrite statements?**

This utility will be used during the transition from structured OTTL
statements to flat statements.
Components such as the `transformprocessor` will leverage it to support
both configuration styles, without forcing
users to adapt/rewrite their existing config files. 

Once the component turns on the `ottl.Parser[K]` path's context
validation, new configuration style usages will be validated, requiring
all paths to have a context prefix, and old configuration styles will
automatically rewrite the statements using this function.

For more details, please have a look at the complete
[draft](open-telemetry#35050)
implementation.

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue

open-telemetry#29017

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Unit tests

<!--Describe the documentation added.-->
#### Documentation
No changes

<!--Please delete paragraphs that you did not use before submitting.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants