[exporter/awsemfexporter] Add config option to retain the initial value of delta metrics (#16218) #17988

mircohacker · 2023-01-24T09:08:03Z

Description:
Some metrics (i.e. Counter) are cumulative in the data model of open telemetry, but AWS EMF expects delta values as contents of the value field. Previously this was solved by dropping the first occurrence of an specific metric label combination and only storing its value as baseline to calculate the difference to the next occurrence of this metric. This is fine for metrics where the value changes a lot and exactness is not important (i.e. requests{code=200}). But there are use cases where a metric should track events which do not occur often or have a high cardinality in its labels. In this case the current implementation drops a significant percentage of data. With this PR merged the new configuration flag retain_initial_value_of_delta_metric is introduced which changes this behavior. The implementation still tries to compute the delta but when no previous value is found, the received value is then assumed to be the delta value or more exact the previous value is assumed to be 0 or 0 equivalent for summaries.

This PR does not change the behavior of the exporter in default configuration.

Suggestions for a better name are gladly accepted but this was the best i could come up with.

Link to tracking Issue:

Testing:
Edit test datapoint_test.go to test both configuration valued for the new flag with expected output. The default case stayed the same and only the behavior for the flag set to true changed. The GitHub diff does not recognize that on most lines only the indention changed...
Also:

make test benchmark in dir exporter/awsemfexporter
make common in repo root
make gofmt in repo root

Documentation: Documentation was added in Readme, next to the config value as code comments and as changelog entry

linux-foundation-easycla · 2023-01-24T09:08:07Z

The committers listed above are authorized under a signed CLA.

✅ login: mircohaug / name: Mirco Hacker (01de021b2f89a73e3b7f996648404e10c96f985b)

runforesight · 2023-01-26T05:05:55Z

Foresight Summary

Major Impacts
Foresight hasn't detected any major impact on your workflows and tests.

View More Details

✅ tracegen workflow has finished in 1 minute 3 seconds (1 minute 22 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

Job	Failed Steps	Tests
build-dev	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details
publish-latest	- 🔗	N/A	See Details

⭕ build-and-test-windows workflow has finished in 8 seconds (43 minutes 7 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

Job	Failed Steps	Tests
windows-unittest-matrix	- 🔗	N/A	See Details
windows-unittest	- 🔗	N/A	See Details

✅ check-links workflow has finished in 1 minute 37 seconds (57 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

Job	Failed Steps	Tests
changed files	- 🔗	N/A	See Details
check-links	- 🔗	N/A	See Details

✅ telemetrygen workflow has finished in 1 minute 33 seconds (1 minute 39 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

Job	Failed Steps	Tests
build-dev	- 🔗	N/A	See Details
publish-latest	- 🔗	N/A	See Details
publish-stable	- 🔗	N/A	See Details

✅ changelog workflow has finished in 2 minutes 31 seconds and finished at 1st Mar, 2023.

Job	Failed Steps	Tests
changelog	- 🔗	N/A	See Details

build-and-test workflow has finished in 15 minutes 3 seconds (57 minutes 4 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

Job	Failed Steps	Tests
unittest-matrix (1.20, internal)	N/A	✅ 581 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, internal)	N/A	✅ 581 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, processor)	N/A	✅ 1526 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, extension)	N/A	✅ 537 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.20, extension)	N/A	✅ 537 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.20, processor)	N/A	✅ 1526 ❌ 0 ⏭ 0 🔗	See Details
correctness-metrics	N/A	✅ 2 ❌ 0 ⏭ 0 🔗	See Details
correctness-traces	N/A	✅ 17 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, receiver-0)	N/A	✅ 2580 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.20, receiver-0)	N/A	✅ 2580 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.20, exporter)	N/A	✅ 2465 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, exporter)	N/A	✅ 2465 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.20, receiver-1)	N/A	✅ 1937 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.20, other)	N/A	✅ 4788 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, other)	N/A	✅ 4788 ❌ 0 ⏭ 0 🔗	See Details
unittest-matrix (1.19, receiver-1)	N/A	✅ 1937 ❌ 0 ⏭ 0 🔗	See Details
integration-tests	N/A	✅ 55 ❌ 0 ⏭ 0 🔗	See Details

✅ prometheus-compliance-tests workflow has finished in 3 minutes 28 seconds (5 minutes 44 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

Job	Failed Steps	Tests
prometheus-compliance-tests	- 🔗	✅ 21 ❌ 0 ⏭ 0 🔗	See Details

✅ load-tests workflow has finished in 7 minutes 1 second (10 minutes 39 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

Job	Failed Steps	Tests
loadtest (TestTraceAttributesProcessor)	- 🔗	✅ 3 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestIdleMode)	- 🔗	✅ 1 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestMetric10kDPS\|TestMetricsFromFile)	- 🔗	✅ 6 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceNoBackend10kSPS\|TestTrace1kSPSWithAttrs)	- 🔗	✅ 8 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestTraceBallast1kSPSWithAttrs\|TestTraceBallast1kSPSAddAttrs)	- 🔗	✅ 10 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestMetricResourceProcessor\|TestTrace10kSPS)	- 🔗	✅ 12 ❌ 0 ⏭ 0 🔗	See Details
loadtest (TestBallastMemory\|TestLog10kDPS)	- 🔗	✅ 18 ❌ 0 ⏭ 0 🔗	See Details
setup-environment	- 🔗	N/A	See Details

✅ e2e-tests workflow has finished in 11 minutes 47 seconds (4 minutes 54 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

Job	Failed Steps	Tests
kubernetes-test (v1.26.0)	- 🔗	N/A	See Details
kubernetes-test (v1.25.3)	- 🔗	N/A	See Details
kubernetes-test (v1.24.7)	- 🔗	N/A	See Details
kubernetes-test (v1.23.13)	- 🔗	N/A	See Details

🔎 See details on Foresight

^{*You can configure Foresight comments in your organization settings page.}

MovieStoreGuy · 2023-01-27T10:31:20Z

I don't fully grasp the implication of this, if there is any negative impact adding this in.

As code owner, can I ask you to double check @Aneurysm9 ?

MovieStoreGuy · 2023-02-06T22:29:41Z

Can I ask you to rebase against main @mircohaug ?

Aneurysm9 · 2023-02-07T01:16:10Z

I don't fully grasp the implication of this, if there is any negative impact adding this in.

As code owner, can I ask you to double check @Aneurysm9 ?

I don't think that there are any unavoidable negative side effects of doing this, particularly since it is opt-in, requiring the user to make a configuration change to accept this behavior. I'm not 100% sure why it was done this way in the first place and Chesterton's Fence makes me want to keep the default behavior as-is.

MovieStoreGuy · 2023-02-08T03:56:25Z

Can I ask you to fix up the merge conflicts @mircohaug ?

mircohacker · 2023-02-08T19:49:57Z

@MovieStoreGuy I just rebased the branch. This configuration option is intended to be used in a scenario where the collector is deployed as a sidecar to one app instance. When the collector is deployed independently of the app, then it will happen, that the first value seen by the collector is not the first value sent by the app. So the collector sees requests{code=200} 123 and sends 123 as delta value to AWS EMF when in reality only a few requests are "new". This could trigger all kinds of downstream systems analyzing this metric. In the default configuration this new value would just get ignored and stored as new delta baseline.

@Aneurysm9 I think that this is also the reason it was done this way from the beginning. AWS EMF works on deltas whereas otel works with monotone increasing counters. At some point the conversion has to be made and it is unavoidable to "loose" some information one way or the other.

MovieStoreGuy · 2023-02-08T23:50:16Z

Just some linting issues :)

I believe you just need to run:

make fmt

In the directory.

TylerHelmuth

@mircohaug please review the lint failures.

mircohacker · 2023-02-11T19:18:31Z

Some merge hiccups 🤷 should be fixed now

MovieStoreGuy · 2023-02-21T10:28:44Z

@mircohaug , could I ask you rebase main again?

mircohacker · 2023-02-24T13:32:15Z

@MovieStoreGuy done

codeboten

Looks like another change snuck in to create a conflict :(

mircohacker · 2023-02-28T09:33:17Z

@codeboten @MovieStoreGuy Rebased it is. It is also kind of tedious...

codeboten · 2023-02-28T17:53:43Z

@codeboten @MovieStoreGuy Rebased it is. It is also kind of tedious...

💯 agree. Will try to get this pr merge asap. It looks like the change failed tests in the awsemfexporter:

 === RUN   TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_1st_delta_sum_count_calculation_retainInitialValueOfDeltaMetric=true
    datapoint_test.go:435: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/awsemfexporter/datapoint_test.go:435
        	Error:      	Max difference between 18 and 17 allowed is 0.002, but difference was 1
        	Test:       	TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_1st_delta_sum_count_calculation_retainInitialValueOfDeltaMetric=true
=== RUN   TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_2nd_delta_sum_count_calculation
=== RUN   TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_3rd_delta_sum_count_calculation
=== RUN   TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_1st_delta_sum_count_calculation_retainInitialValueOfDeltaMetric=false
=== RUN   TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_2nd_delta_sum_count_calculation#01
=== RUN   TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_3rd_delta_sum_count_calculation#01
--- FAIL: TestCalculateDeltaDatapoints_SummaryDataPointSlice (0.00s)
    --- FAIL: TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_1st_delta_sum_count_calculation_retainInitialValueOfDeltaMetric=true (0.00s)
    --- PASS: TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_2nd_delta_sum_count_calculation (0.00s)
    --- PASS: TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_3rd_delta_sum_count_calculation (0.00s)
    --- PASS: TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_1st_delta_sum_count_calculation_retainInitialValueOfDeltaMetric=false (0.00s)
    --- PASS: TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_2nd_delta_sum_count_calculation#01 (0.00s)
    --- PASS: TestCalculateDeltaDatapoints_SummaryDataPointSlice/Detailed_summary_with_3rd_delta_sum_count_calculation#01 (0.00s)

…ue of delta metrics (open-telemetry#16218)

mircohacker · 2023-03-01T20:25:37Z

My bad. I wanted to see the tests red as well and forgot to make this one green again 🙈 @codeboten @MovieStoreGuy Im asking you once again to merge this PR 😉

…o translateOTelToGroupedMetric (#24051) **Description:** The config option `retain_initial_value_of_delta_metric` does not seem to be used in `translateOTelToGroupedMetric`, which prevents the initial value of a basic counter from being published during a Lambda cold boot. Please see the minimum project required to replicate the issue [here](https://github.com/jameshi16/delta-initial-value-minimum-project). **Link to tracking Issue:** The main issue related to this PR can be found [here](aws-observability/aws-otel-lambda#634). It seems like I had a predecessor fixing this issue (see #17988), but his changes does not work for my use case. **Testing:** An additional test ensures that if `retain_initial_value_of_delta_metric` is set, it will be propagated to the `cWMetricMetadata`. **Documentation:** None

* [chore] dependabot updates Wed Aug 9 04:37:47 UTC 2023 (#25105) Bump github.com/Azure/azure-event-hubs-go/v3 from 3.6.0 to 3.6.1 in /receiver/azureblobreceiver Bump github.com/Azure/azure-event-hubs-go/v3 from 3.6.0 to 3.6.1 in /receiver/azureeventhubreceiver Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /exporter/awscloudwatchlogsexporter Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /exporter/awsemfexporter Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /exporter/awsxrayexporter Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /exporter/kafkaexporter Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /extension/observer/ecsobserver Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /internal/aws/awsutil Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /internal/aws/cwlogs Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /internal/aws/k8s Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /internal/aws/proxy Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /internal/aws/xray Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /internal/aws/xray/testdata/sampleapp Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /internal/metadataproviders Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /processor/resourcedetectionprocessor Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /receiver/awscontainerinsightreceiver Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /receiver/awsecscontainermetricsreceiver Bump github.com/aws/aws-sdk-go from 1.44.316 to 1.44.319 in /receiver/awsxrayreceiver Bump github.com/aws/aws-sdk-go-v2 from 1.20.0 to 1.20.1 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/config from 1.18.32 to 1.18.33 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.31 to 1.13.32 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/service/sts from 1.21.1 to 1.21.2 in /extension/sigv4authextension Bump github.com/getsentry/sentry-go from 0.22.0 to 0.23.0 in /exporter/sentryexporter Bump github.com/hashicorp/consul/api from 1.23.0 to 1.24.0 in /internal/metadataproviders Bump github.com/hashicorp/consul/api from 1.23.0 to 1.24.0 in /processor/resourcedetectionprocessor Bump github.com/shirou/gopsutil/v3 from 3.23.6 to 3.23.7 in /exporter/signalfxexporter Bump github.com/shirou/gopsutil/v3 from 3.23.6 to 3.23.7 in /extension/observer/hostobserver Bump github.com/shirou/gopsutil/v3 from 3.23.6 to 3.23.7 in /receiver/awscontainerinsightreceiver Bump github.com/shirou/gopsutil/v3 from 3.23.6 to 3.23.7 in /receiver/hostmetricsreceiver Bump github.com/shirou/gopsutil/v3 from 3.23.6 to 3.23.7 in /receiver/jmxreceiver Bump github.com/shirou/gopsutil/v3 from 3.23.6 to 3.23.7 in /testbed Bump github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common from 1.0.715 to 1.0.717 in /exporter/tencentcloudlogserviceexporter Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /extension/observer/dockerobserver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /internal/coreinternal Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/aerospikereceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/apachereceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/dockerstatsreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/elasticsearchreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/flinkmetricsreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/jmxreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/kafkametricsreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/memcachedreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/mongodbreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/mysqlreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/nginxreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/postgresqlreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/redisreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/sqlqueryreceiver Bump github.com/testcontainers/testcontainers-go from 0.21.0 to 0.22.0 in /receiver/zookeeperreceiver Bump github.com/vmware/govmomi from 0.30.6 to 0.30.7 in /receiver/vcenterreceiver Bump go.mongodb.org/mongo-driver from 1.12.0 to 1.12.1 in /receiver/mongodbreceiver Bump golang.org/x/crypto from 0.11.0 to 0.12.0 in /receiver/podmanreceiver Bump golang.org/x/net from 0.12.0 to 0.14.0 in /exporter/azuremonitorexporter Bump golang.org/x/net from 0.12.0 to 0.14.0 in /internal/aws/awsutil Bump golang.org/x/oauth2 from 0.10.0 to 0.11.0 in /exporter/f5cloudexporter Bump golang.org/x/oauth2 from 0.10.0 to 0.11.0 in /extension/oauth2clientauthextension Bump golang.org/x/sys from 0.10.0 to 0.11.0 in /exporter/signalfxexporter Bump golang.org/x/sys from 0.10.0 to 0.11.0 in /pkg/stanza Bump golang.org/x/sys from 0.10.0 to 0.11.0 in /pkg/winperfcounters Bump golang.org/x/sys from 0.10.0 to 0.11.0 in /receiver/hostmetricsreceiver Bump golang.org/x/sys from 0.10.0 to 0.11.0 in /receiver/windowseventlogreceiver Bump golang.org/x/text from 0.11.0 to 0.12.0 in /cmd/configschema Bump golang.org/x/text from 0.11.0 to 0.12.0 in /cmd/mdatagen Bump golang.org/x/text from 0.11.0 to 0.12.0 in /internal/coreinternal Bump golang.org/x/text from 0.11.0 to 0.12.0 in /pkg/stanza Bump golang.org/x/text from 0.11.0 to 0.12.0 in /testbed Bump golang.org/x/tools from 0.11.1 to 0.12.0 in /internal/tools Bump sigs.k8s.io/controller-runtime from 0.15.0 to 0.15.1 in /exporter/loadbalancingexporter --------- Co-authored-by: Dmitrii Anoshin <[email protected]> * [chore] preallocate slices (#24957) * [chore] dependabot updates Wed Aug 9 06:51:47 UTC 2023 (#25113) Bump github.com/hashicorp/golang-lru from 0.5.4 to 1.0.2 in /extension/observer/ecsobserver Bump github.com/hashicorp/golang-lru from 0.6.0 to 1.0.2 in /processor/spanmetricsprocessor Bump github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common from 1.0.717 to 1.0.719 in /exporter/tencentcloudlogserviceexporter Bump google.golang.org/api from 0.134.0 to 0.136.0 in /receiver/googlecloudspannerreceiver * [exporter/clickhouse] Change the type of `Config.Password` to be `configopaque.String` (#24778) clickhouse exporter also needs use `Config.Password`. --------- Co-authored-by: Dmitrii Anoshin <[email protected]> * [chore] Add a new Test and refactor code function for collectdreceiver (#25120) **Description:** Add a new test 'testInvalidMethod' in receiver_test.go, and refactor code for readability. * [receiver/hostmetrics] Improve description of system.cpu.utilization metric (#25123) Replaces https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/25115 --------- Co-authored-by: Fabrizio Ferri-Benedetti <[email protected]> Co-authored-by: Ryan Fitzpatrick <[email protected]> * [chore] [exporter/signalfx] Add few tests for histogram metrics (#25127) **Description:** [exporter/signalfx] Added few tests for histogram metrics on SFx exporter's converter piece **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25067 * [chore] [processor/transform] enable exhaustive lint (#25140) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> * [chore][exporter/clickhouse]enable exhaustive for linter (#25133) relate https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23266 * [chore][exporter/azuredataexplore]enable exhaustive linter (#25130) relate https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23266 * [chore] [dataset/exporter]: Fix flaky windows tests (#25143) **Description:** There is flaky Windows test **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25094 **Testing:** Before applying this change, it has failed in 1 test out of 60. So this should pass 200 tests without failure. * [chore][exporter/sumologic] enable exhaustive linter (#25159) relate: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23266 * [chore][exporter/skywalking] enable exhaustive linter (#25158) relate: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23266 * [chore][exporter/googlecloudpubsub] enable exhaustive linter (#25156) relate https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23266 * [exporter/awsemfexporter] Propagate RetainInitialValueOfDeltaMetric to translateOTelToGroupedMetric (#24051) **Description:** The config option `retain_initial_value_of_delta_metric` does not seem to be used in `translateOTelToGroupedMetric`, which prevents the initial value of a basic counter from being published during a Lambda cold boot. Please see the minimum project required to replicate the issue [here](https://github.com/jameshi16/delta-initial-value-minimum-project). **Link to tracking Issue:** The main issue related to this PR can be found [here](https://github.com/aws-observability/aws-otel-lambda/issues/634). It seems like I had a predecessor fixing this issue (see #17988), but his changes does not work for my use case. **Testing:** An additional test ensures that if `retain_initial_value_of_delta_metric` is set, it will be propagated to the `cWMetricMetadata`. **Documentation:** None * [chore][exporter/dynatrace] enable exhaustive linter (#25155) relate https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23266 * [chore] [CI] Exclude go version matrix from the unittest job (#25151) To not update required jobs every time a new go version is released. Needed for https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25116 * Bump filelog.allowHeaderMetadataParsing feature gate to beta (#25173) * [processor/k8sattributes] Add optional k8s.cluster.uid resource attribute (#23668) **Description:** Add k8s.cluster.uid to attribute to k8sattributes processor and disable it by default for backward compatibility. Users can set it to `true` to populate cluster uid as part of resource attributes. **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/21974 * [chore] Upgrade koanf to v2 in the Supervisor (#25169) This is primarily to align the version with what the Collector uses in `confmap`. It doesn't introduce any functional changes. * Remove go 1.19 support, bump minimum to go 1.20 and add testing for 1.21 (#25162) Fixes https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25116 * [chore] [receiver/file] enable exhaustive lint (#25180) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> * [chore] [exporter/awskinesis] Migrate from deprecated kinesis.WithEndpointResolver (#25164) Fixes https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25104 * [chore] dependabot updates Sun Aug 13 19:41:07 UTC 2023 (#25182) Bump github.com/openzipkin/zipkin-go from 0.4.1 to 0.4.2 in /receiver/pulsarreceiver * [receiver/sqlquery] set ObservedTimestamp on collected logs (#23777) Fixes #23776 * [chore] [clickhouseexporter] use errors.Join instead of go.uber.org/multierr (#25183) Part of: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25121 * [chore] migrate all the files under connector to use errors.Join (#25184) Related to https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25121 * Datadog Connector Component (#25065) This pull request is for a newly created Datadog connector that replaces the need for the Datadog processor. This replacement is needed because the processor made use of a work-around that directly allowed the processor to export data. This is not something that adheres to the OpenTelemetry standard and thus the previous system should be deprecated and replaced by this new connector. Link to issue: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/19740 --------- Co-authored-by: Yang Song <[email protected]> Co-authored-by: Pablo Baeyens <[email protected]> * [chore] Migrate all `aws` receviers to use errors.Join (#25185) Related to https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25121 * [pkg/stanza] Create a new decoder for each TCP/UDP connection to prevent concurrent write to buffer. (#25100) Change the TCP/UDP inputs so that they parse the encoding and get the `encoding.Encoding` during initialization, but wait to create the `helper.Encoding` instance in each goroutine. This prevents concurrent calls to `Decode` using the same instance. Co-authored-by: Dan Jaglowski <[email protected]> * [extension/jaegerremotesampling] Support reload_interval option in remote mode of usage (#24981) Updates the jaegerremotesampling extension's `remote` mode of usage to support the `reload_interval` caching option already supported in `file` mode of usage. Co-authored-by: Anthony Mirabella <[email protected]> * [translator/loki] Fixes a panic that occurred during the promotion of nested attributes containing dots to labels (#25142) When the value for loki hint attribute contains dots (more than 1 dot) and attributes are nested so some level contains dotted attribute name (for example `{"log": {"file.name": "foo"}}`), the getNestedAttribute function throws a panic Co-authored-by: Tyler Helmuth <[email protected]> * [chore] [receiver/sqlquery] Fix flaky test TestLogsQueryReceiver_Collect (#25191) Fixes https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25189 * Prepare update to core v0.83.0 (#25187) Corresponds to [Step 3](https://github.com/open-telemetry/opentelemetry-collector/blob/81242fa7eff332756df6e03034c0e6686b9b988e/docs/release.md?plain=1#L34) of collector core release process. Updated as part of Step 8. --------- Signed-off-by: Alex Boten <[email protected]> Co-authored-by: Alex Boten <[email protected]> * [chore] dependabot updates Mon Aug 14 23:07:29 UTC 2023 (#25810) Bump github.com/ClickHouse/clickhouse-go/v2 from 2.12.1 to 2.13.0 in /exporter/clickhouseexporter Bump github.com/antonmedv/expr from 1.12.5 to 1.13.0 in /receiver/receivercreator Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.322 in /exporter/kafkaexporter Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.322 in /internal/aws/k8s Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.322 in /internal/aws/xray/testdata/sampleapp Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.322 in /receiver/awscontainerinsightreceiver Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.322 in /receiver/awsxrayreceiver Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /exporter/awscloudwatchlogsexporter Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /exporter/awsxrayexporter Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /exporter/kafkaexporter Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /internal/aws/k8s Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /internal/aws/proxy Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /internal/aws/xray Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /internal/aws/xray/testdata/sampleapp Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /receiver/awscontainerinsightreceiver Bump github.com/aws/aws-sdk-go from 1.44.319 to 1.44.323 in /receiver/awsxrayreceiver Bump github.com/daixiang0/gci from 0.10.1 to 0.11.0 in /internal/tools Bump github.com/golangci/golangci-lint from 1.53.3 to 1.54.1 in /internal/tools Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /extension/observer/dockerobserver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /internal/coreinternal Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/aerospikereceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/apachereceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/dockerstatsreceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/elasticsearchreceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/jmxreceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/kafkametricsreceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/nginxreceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/postgresqlreceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/redisreceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/sqlqueryreceiver Bump github.com/testcontainers/testcontainers-go from 0.22.0 to 0.23.0 in /receiver/zookeeperreceiver Bump go.mongodb.org/atlas from 0.32.0 to 0.33.0 in /receiver/mongodbatlasreceiver Bump go.opentelemetry.io/collector from 0.82.0 to 0.83.0 in /pkg/stanza Bump go.opentelemetry.io/collector from 0.82.0 to 0.83.0 in /receiver/carbonreceiver Bump go.opentelemetry.io/collector/cmd/builder from 0.82.1-0.20230811173841-81242fa7eff3 to 0.83.0 in /internal/tools Bump go.opentelemetry.io/collector/component from 0.82.0 to 0.83.0 in /exporter/logzioexporter Bump go.opentelemetry.io/collector/config/confighttp from 0.82.0 to 0.83.0 in /receiver/elasticsearchreceiver Bump go.opentelemetry.io/collector/config/confighttp from 0.82.0 to 0.83.0 in /receiver/skywalkingreceiver Bump go.opentelemetry.io/collector/config/confignet from 0.82.0 to 0.83.0 in /extension/jaegerremotesampling Bump go.opentelemetry.io/collector/config/configopaque from 0.82.0 to 0.83.0 in /extension/jaegerremotesampling Bump go.opentelemetry.io/collector/config/configtls from 0.82.0 to 0.83.0 in /processor/resourcedetectionprocessor Bump go.opentelemetry.io/collector/config/configtls from 0.82.0 to 0.83.0 in /receiver/sapmreceiver Bump go.opentelemetry.io/collector/config/configtls from 0.82.0 to 0.83.0 in /receiver/signalfxreceiver Bump go.opentelemetry.io/collector/confmap from 0.82.0 to 0.83.0 in /exporter/splunkhecexporter Bump go.opentelemetry.io/collector/confmap from 0.82.0 to 0.83.0 in /processor/logstransformprocessor Bump go.opentelemetry.io/collector/confmap from 0.82.0 to 0.83.0 in /receiver/awsfirehosereceiver Bump go.opentelemetry.io/collector/confmap from 0.82.0 to 0.83.0 in /receiver/rabbitmqreceiver Bump go.opentelemetry.io/collector/confmap from 0.82.0 to 0.83.0 in /receiver/saphanareceiver Bump go.opentelemetry.io/collector/confmap from 0.82.0 to 0.83.0 in /receiver/signalfxreceiver Bump go.opentelemetry.io/collector/consumer from 0.82.0 to 0.83.0 in /exporter/opencensusexporter Bump go.opentelemetry.io/collector/consumer from 0.82.0 to 0.83.0 in /processor/logstransformprocessor Bump go.opentelemetry.io/collector/consumer from 0.82.0 to 0.83.0 in /processor/resourcedetectionprocessor Bump go.opentelemetry.io/collector/consumer from 0.82.0 to 0.83.0 in /receiver/mysqlreceiver Bump go.opentelemetry.io/collector/consumer from 0.82.0 to 0.83.0 in /receiver/nsxtreceiver Bump go.opentelemetry.io/collector/processor from 0.82.0 to 0.83.0 in /exporter/datadogexporter Bump go.opentelemetry.io/collector/receiver from 0.82.0 to 0.83.0 in /receiver/dockerstatsreceiver Bump go.opentelemetry.io/collector/receiver from 0.82.0 to 0.83.0 in /receiver/googlecloudpubsubreceiver Bump go.opentelemetry.io/collector/semconv from 0.82.0 to 0.83.0 in /exporter/awsemfexporter Bump go.opentelemetry.io/collector/semconv from 0.82.0 to 0.83.0 in /receiver/hostmetricsreceiver * [chore] Prepare release 0.83.0 (#25819) The following commands were run to prepare this release: - make chlog-update VERSION=v0.83.0 - sed -i.bak s/0.82.0/0.83.0/g versions.yaml - make multimod-prerelease - make multimod-sync * [chore] Move a few entries between changelogs (#25822) Followup to the v0.83.0 release. The chloggen tool was recently upgraded to support multiple separate changelogs. While the tool worked correctly, several entries were added prior to the new feature being added, resulting in several entries going to the default (user) changelog rather than the api changelog. * [chore] [receiver/k8scluster] enable exhaustive lint (#25818) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> * [processor/k8sattributes] Move config validation into Validate (#25153) **Description:** Moves existing config validation into the Validate function **Testing:** Added more unit tests --------- Co-authored-by: Dmitrii Anoshin <[email protected]> * Update otel/opentelemetry-collector-contrib Docker tag to v0.83.0 (#25826) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [otel/opentelemetry-collector-contrib](https://togithub.com/open-telemetry/opentelemetry-collector-releases) | minor | `0.82.0` -> `0.83.0` | --- ### Release Notes <details> <summary>open-telemetry/opentelemetry-collector-releases (otel/opentelemetry-collector-contrib)</summary> ### [`v0.83.0`](https://togithub.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.83.0) [Compare Source](https://togithub.com/open-telemetry/opentelemetry-collector-releases/compare/v0.82.0...v0.83.0) #### Changelog - [`66a5e00`](https://togithub.com/open-telemetry/opentelemetry-collector-releases/commit/66a5e00) Prep release v0.83.0 ([#388](https://togithub.com/open-telemetry/opentelemetry-collector-releases/issues/388)) - [`8eb6a21`](https://togithub.com/open-telemetry/opentelemetry-collector-releases/commit/8eb6a21) Bump github.com/goreleaser/goreleaser from 1.19.2 to 1.20.0 ([#387](https://togithub.com/open-telemetry/opentelemetry-collector-releases/issues/387)) - [`9157d9d`](https://togithub.com/open-telemetry/opentelemetry-collector-releases/commit/9157d9d) \[admin] Add criteria for distributions ([#376](https://togithub.com/open-telemetry/opentelemetry-collector-releases/issues/376)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib).  * [chore] [cmd/mdatagen] Fix a comment in resource_test.go (#25827) Cutting it from https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/24444 * [chore] [receiver/haproxy] Use generated pdata output in the test (#25824) Cutting it from https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/24444 to reduce the change set * [chore][fileconsumer] Move splitter factory code into internal package (#25172) * [receiver/influxdb] Fix time precision parser (#24974) (#25791) Add precision=n and precision=u to line protocol endpoint handler. This explicitly matches the behavior of InfluxDB 1.11, and allows a certain [Java client](https://github.com/influxdata/influxdb-java/blob/master/src/main/java/org/influxdb/impl/TimeUtil.java#L63) to write to OpenTelemetry. Also adds precision=µ and precision=µs for more complete compatibility with github.com/influxdata/line-protocol . * [receiver/postgresql] Added postgresql.deadlocks metric. (#25792) **Description:** Added new metric -- `postgresql.deadlocks` -- number of deadlocks detected in a database. **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25688 * [chore] [receiver/postgres] Remove custom metrics builder code (#25830) The code was introduced to migrate table and database datapoint attributes to resource attributes in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/12967. Migration was completed a long time ago. The custom code isn't needed anymore. This also fixed the documentation to match the emitted data and removes redundant testdata * [chore] update segment download timeout to 2 minutes across all builds (#25832) Fixes #25794 Fixes #21506 * [receiver/vcenter] Metric attributes for host and vm performance metrics (#25149) Adds 'object' data point attribute to several metrics which has undifferentiated data points * [chore] cache telemetrygen image to avoid test flakiness (#25825) Fixes #24223 * [chore] update rotation (#25841) @TylerHelmuth sponsored vendor component here: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/24294 Signed-off-by: Alex Boten <[email protected]> * [chore] make tests faster and close http responses (#24925) These changes only touch tests. * [cmd/telemetrygen] Add status code to cli argument (#24673) **Description:** Adding command line argument `--status-code` to `telemetrygen traces`, which accepts `(Unset,Error,Ok)` (case sensitive) or the enum equivalent of `(0,1,2)`. Running ```shell telemetrygen traces --otlp-insecure --traces 1 --status-code 1 ``` against a minimal local collector yields ```txt 2023-07-29T21:27:57.862+0100 info ResourceSpans #0 Resource SchemaURL: https://opentelemetry.io/schemas/1.4.0 Resource attributes: -> service.name: Str(telemetrygen) ScopeSpans #0 ScopeSpans SchemaURL: InstrumentationScope telemetrygen Span #0 Trace ID : f6dc4be32c78b9999c69d504a79e68c1 Parent ID : 4e2cd6e0e90cf2ea ID : 20835413e32d26a5 Name : okey-dokey Kind : Server Start time : 2023-07-29 20:27:57.861602 +0000 UTC End time : 2023-07-29 20:27:57.861726 +0000 UTC Status code : Error Status message : Attributes: -> net.peer.ip: Str(1.2.3.4) -> peer.service: Str(telemetrygen-client) Span #1 Trace ID : f6dc4be32c78b9999c69d504a79e68c1 Parent ID : ID : 4e2cd6e0e90cf2ea Name : lets-go Kind : Client Start time : 2023-07-29 20:27:57.861584 +0000 UTC End time : 2023-07-29 20:27:57.861726 +0000 UTC Status code : Error Status message : Attributes: -> net.peer.ip: Str(1.2.3.4) -> peer.service: Str(telemetrygen-server) ``` and similarly (the string version) ```shell telemetrygen traces --otlp-insecure --traces 1 --status-code '"Ok"' ``` produces ```txt Resource SchemaURL: https://opentelemetry.io/schemas/1.4.0 Resource attributes: -> service.name: Str(telemetrygen) ScopeSpans #0 ScopeSpans SchemaURL: InstrumentationScope telemetrygen Span #0 Trace ID : dfd830da170acfe567b12f87685d7917 Parent ID : 8e15b390dc6a1ccc ID : 165c300130532072 Name : okey-dokey Kind : Server Start time : 2023-07-29 20:29:16.026965 +0000 UTC End time : 2023-07-29 20:29:16.027089 +0000 UTC Status code : Ok Status message : Attributes: -> net.peer.ip: Str(1.2.3.4) -> peer.service: Str(telemetrygen-client) Span #1 Trace ID : dfd830da170acfe567b12f87685d7917 Parent ID : ID : 8e15b390dc6a1ccc Name : lets-go Kind : Client Start time : 2023-07-29 20:29:16.026956 +0000 UTC End time : 2023-07-29 20:29:16.027089 +0000 UTC Status code : Ok Status message : Attributes: -> net.peer.ip: Str(1.2.3.4) -> peer.service: Str(telemetrygen-server) ``` The default is `Unset` which is the current behaviour. **Link to tracking Issue:** 24286 **Testing:** Added unit tests which covers both valid and invalid inputs. **Documentation:** Command line arguments are self documenting via the usage info in the flag. Co-authored-by: Pablo Baeyens <[email protected]> * [receiver/vcenter] Add resource attribute for resource pool inventory path (#25842) ix the issue where the integration test was emitting resource pools resources with the same identifiers. A resource pool generally isn't going to be named the same unless there is some nested structure and generally is believed to be a malpractice from my experience. The inventory path may be the best all around solution for adding context to where the metrics are coming from. * [chore][fileconsumer] Extract a readerMetadata struct (#25828) This PR continues the incremental refactoring of the fileconsumer package. I'd like to eventually move the reader struct into an internal package. Prior to doing so, I believe it will be helpful to simplify the struct's contents and usage. - Extract a readerMetadata struct from the reader struct. This contains exactly the fields which can be saved and reloaded. Consequently, saving and reloading will operator directly on the metadata, rather than the entire reader. - Simplify the readerFactory and readerBuilder by pulling some nuanced responsibilities out of the builder and into the factory's specialized functions. - Remove several builder options, which are now handled by factory. - Remove special case where fingerprint was initialized by builder. This was only used in tests. - Remove useless test which only validates that json encoder functionality. * [chore][golden] Normal timestamps in golden metric files (#23002) Normalized the timestamps on Golden file outputs. Normalization in this instance basically means setting non-unique timestamps of a metric equal to each other and unique timestamps incremented in increments of 1,000,000 nanoseconds. To see a clear example of this, navigate into the added test files (internal/coreinternal/golden/testdata/timestamp-norm) The goal is to further make the Golden output files as deterministic as possible and clear up the cluttered git diff's every time one of these golden files are remade. * [chore] update codeowners to respect capitalization (#25848) Update github logins to match capitalization. * [pkg/stanza] Deprecate EncodingConfig (#25846) * Revert "[chore] cache telemetrygen image to avoid test flakiness" (#25853) Reverts open-telemetry/opentelemetry-collector-contrib#25825 * [chore] do not ignore error when json marshaling (#24934) * [exporter/opensearch] Send traces to OpenSearch (#24540) Implementation of exporter to OpenSearch using opensearch-go library. Requests sent adhere to OpenSearch Catalog [schema for traces](https://github.com/opensearch-project/opensearch-catalog/tree/main/schema/observability/traces). # Testing ## Integration - Successful round-trip to HTTP endpoint, - Permanent error during round-trip, - Retryable error response for first request, followed by successful response on retry, - Two retriable error responses, followed by successful response on second retry. ## Manual end-to-end - Authenticating using basicauth extension, - Connecting to OpenSearch using TLS. ## Issues Found ### Rejected Attribute JSON In bdd41142467997c3261c5c991ddffd038fff43db, some values in `ptrace.Span.Attributes()` can be converted to JSON that OpenSearch Observability's ss4o index template rejects. The collector correctly logs these as dropped. Working with OpenSearch observability to find root cause. The fix will be localized to `sso_model.go` and `traceBulkIndexer.createJSON`. --------- Signed-off-by: Max Ksyunz <[email protected]> Co-authored-by: Anthony Mirabella <[email protected]> * [chore] prealloc for internal/coreinternal (#24947) * [chore] Code linter findings for internal/aws (#24946) preallocate slices do not assign a boolean if it is reassigned right after * [chore] Add TylerHelmuth to k8s components codeowners (#25869) **Description:** I'd like to be more actively involved with the collector's k8s components. * [chore] [pkg/stanza] correct code comment (#25873) Corrects an inaccurate comment header on the FromPdataConverter type. **Description:** Small correction to a go comment that threw me for a loop. Thanks! **Link to tracking Issue:** NA **Testing:** NA **Documentation:** None Signed-off-by: Christian Kruse <[email protected]> * [chore] rename error variable to match expected golang pattern (#24936) * [chore] preallocate slice (#24940) * [chore] preallocate slices throughout the observer extension (#24943) * [chore] [exporter/alibabacloudlogservice] enable exhaustive lint (#25877) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> * Deprecating The Datadog Processor (#25437) The Datadog Processor is deprecated in favor of the Datadog Connector Link to issue: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/19740 --------- Co-authored-by: Yang Song <[email protected]> Co-authored-by: Alex Boten <[email protected]> * [receiver/kubeletstats] Add uptime metric for nodes, pods, and containers (#25867) **Description:** Adds a new monotonic, cumulative sum metric for tracking uptime of nodes, pods, and containers. Uptime is calculated as the number of seconds since the object's `StartTime`. **Testing:** Updated unit tests. Tested locally using the otel demo * [chore][receiver/httpcheck] fix configuration docs (#25881) Replaces `endpoint` with `targets` in README as changed in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/23439. * [chore] Add to TylerHelmuth as codeowner for kubeletstats receiver (#25891) * [chore][exporter/azuremonitor]enable exhaustive for linter (#25131) related to https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23266 Co-authored-by: Alex Boten <[email protected]> * [cmd/telemetrygen] fix status-code args default value (#25889) Fix #25849 after adding more logs for failed telemetrygen pod, I found traces-job keeps failing with the error log ``` Error: expected `status-code` to be one of (Unset, Error, Ok) or (0, 1, 2), got "Unset" instead ``` here is a failed run https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/5899236967/job/16005926116 I thinks this bug is brought in by the pr #24673 , the default value `Unset` for arg `status-code` is not a legal one, the legal one is `"Unset"` --------- Signed-off-by: Ziqi Zhao <[email protected]> * [chore] Remove unused BUILD_INFO from Makefile (#25823) This PR removes unused `BUILD_INFO` from Makefile which was originally deleted in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/16828. As far as I checked briefly, the file itself was already deleted, and no one uses it now. **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/11867 **Testing:** ```shell-session $ make otelcontribcol cd ./cmd/otelcontribcol && GO111MODULE=on CGO_ENABLED=0 go build -trimpath -o ../../bin/otelcontribcol_darwin_arm64 \ -tags "" . $ ./bin/otelcontribcol_darwin_arm64 -v otelcontribcol version 0.83.0-dev ``` Signed-off-by: Kotaro Inoue <[email protected]> * [exporter/azuremonitor] Log exporter handle non-string values (#24911) The log exporter was incorrectly assuming a string value for the body. Minor refactoring of the log exporter unit tests for better isolation. Fixes https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23422 **Testing:** Unit tests --------- Co-authored-by: Alex Boten <[email protected]> * [receiver/haproxy] add support for http (#24560) Support HTTP endpoints as well for haproxy receiver. * [exporter/signalfx] Option to drop histogram buckets. part-1 (#25893) **Description:** : Added a mechanism to drop histogram buckets from the signalfx translator. Please note this will be used in upcoming second part of the "histogram bucket drop" effort for the same issue. **Link to tracking Issue:** #25845 * [receiver/redis] Add username parameter for connecting to redis (#24408) **Description:** Adding a feature - Adding username for redis receiver for connecting to redis over ACL **Testing:** Running go test is ok. Generating my custom OpenTelemetry Collector binary with redis receiver and otlp exporter (using ocb). This custom Opentelemetry Collector collecting redis data by connecting with a username and password work fine. I test to collect data using password only, it works too. **Documentation:** Mardown file has been updated * [receiver/journald]: add support for dmesg (#25910) Adds support for `--dmesg` to journaldreceiver * [chore] [connector/datadog] add status table to readme (#25903) For consistency with other components. Signed-off-by: Alex Boten <[email protected]> * [exporter/opensearch] ensure readme is generated (#25899) This will ensure consistency w/ the other components in the repo. Signed-off-by: Alex Boten <[email protected]> * [receiver/journald]: add support for identifiers (#25911) Adds support for `identifiers` to journaldreceiver * [pkg/ottl] use IntGetter argument for Substring function (#25874) **Description:** <Describe what has changed.>  use IntGetter argument for Substring function **Link to tracking Issue:** <Issue number if applicable> #25852 **Testing:** <Describe what testing was performed and which tests were added.> **Documentation:** <Describe the documentation added.> --------- Co-authored-by: Antoine Toulme <[email protected]> Co-authored-by: Tyler Helmuth <[email protected]> * [internal/filter] enable exhaustive lint (#25908) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> * [internal/coreinternal] enable exhaustive lint (#25907) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> * [exporter/clickhouse] Change writing of metrics data to batch (#24403) **Description:** <Describe what has changed.> The clickhouseexporter using a StringBuilder to build a big sql string while writing metrics. This will be causing the queue to full when metric data is big. Changes: valueArgs := make([]any, e.count*expHistogramValueCounts) var b strings.Builder index := 0 for _, model := range e.expHistogramModels { for i := 0; i < model.expHistogram.DataPoints().Len(); i++ { dp := model.expHistogram.DataPoints().At(i) b.WriteString(*expHistogramPlaceholders) valueArgs[index] = model.metadata.ResAttr valueArgs[index+1] = model.metadata.ResURL ..... valueArgs[index+28] = dp.Max() index += expHistogramValueCounts } } start := time.Now() err := doWithTx(ctx, db, func(tx *sql.Tx) error { _, err := tx.ExecContext(ctx, fmt.Sprintf("%s %s", e.insertSQL, strings.TrimSuffix(b.String(), ",")), valueArgs...) return err }) TO: start := time.Now() err := doWithTx(ctx, db, func(tx *sql.Tx) error { batch, err := tx.PrepareContext(ctx, e.insertSQL) if err != nil { return err } batch: for _, model := range e.expHistogramModels { for i := 0; i < model.expHistogram.DataPoints().Len(); i++ { dp := model.expHistogram.DataPoints().At(i) attrs, times, values, traceIDs, spanIDs := convertExemplars(dp.Exemplars()) _, err = batch.ExecContext(ctx, model.metadata.ResAttr, model.metadata.ResURL, model.metadata.ScopeInstr.Name(), model.metadata.ScopeInstr.Version(), attributesToMap(model.metadata.ScopeInstr.Attributes()), model.metadata.ScopeInstr.DroppedAttributesCount(), model.metadata.ScopeURL, model.metricName, model.metricDescription, model.metricUnit, attributesToMap(dp.Attributes()), dp.StartTimestamp().AsTime(), dp.Timestamp().AsTime(), dp.Count(), dp.Sum(), dp.Scale(), dp.ZeroCount(), dp.Positive().Offset(), convertSliceToArraySet(dp.Positive().BucketCounts().AsRaw()), dp.Negative().Offset(), convertSliceToArraySet(dp.Negative().BucketCounts().AsRaw()), attrs, times, values, traceIDs, spanIDs, uint32(dp.Flags()), dp.Min(), dp.Max(), ) if err != nil { break batch } } } return err }) --------- Co-authored-by: Murphy Chen <[email protected]> * [pkg/ottl] Add new `ExtractPatterns` converter that extract regex patterns from string (#25878) **Description:** <Describe what has changed.>  [pkg/ottl] Add new `ExtractPatterns` converter that extract regex pattern from string **Link to tracking Issue:** <Issue number if applicable> #25834, #25856 **Testing:** <Describe what testing was performed and which tests were added.> Unit tests **Documentation:** <Describe the documentation added.> Added documentation to pkg/ottl/ottlfuncs/README.md. --------- Co-authored-by: Tyler Helmuth <[email protected]> * [chore][fileconsumer] Remove readerBuilder and simplify reader instantiation (#25847) Followup to https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/25828 - Remove the `readerBuilder` entirely and instead works with clearer `readerFactory` methods. - Fixes test setup in `otlpjsonfilereceiver`, which was previous relying on a force flush event. This was exposed as part of this refactoring. * [chore][fileconsumer] Cleanup file handle when closed by reader (#25912) This PR is a small step towards reducing complexity in the way we manage readers and their files. With this change we now have an expectation that reader file handles are always open. * [fileconsumer] Remove deprecated structs (#25916) * Revert "[chore][fileconsumer] Cleanup file handle when closed by reader" (#26021) Reverts open-telemetry/opentelemetry-collector-contrib#25912 This change appears to have caused special behaviors on windows. * Remove retry_on_failure from the googlecloud exporters (#25900) **Description:** In https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/19203, we disabled retry_on_failure by default because retry is now handled by the google cloud client libraries. However, many users still enable it because the error message encourages users to enable retry_on_failure. This is causing problems, especially with metrics because retrying a batch of metrics produces confusing errors (like out-of-order or duplicate timeseries errors) and spams users logs more than necessary. This PR removes the retry_on_failure config from the GMP and googlecloud exporters. * [pkg/translator/loki] Added Attributes support to the InstrumentationScope (#25883) **Description:** <Describe what has changed.> Added Attributes support to InstrumentationScope Now the attributes information in the `pcommon.InstrumentationScope` structure is translated and transmitted to Loki. **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/24027 **Testing:** Added unit tests * [extension/healthcheck] remove dependency on jaeger package (#25895) This extension was pulling in a dependency on jaeger for a fairly small amount of code. I've copied the code into an internal package instead to remove the dep. --------- Signed-off-by: Alex Boten <[email protected]> * Fix replace_all_patterns attributes for transform readme (#26032) In the transform processor readme, there's still one instance of the replace_all_patterns without the mode parameter. This PR fixed that. Signed-off-by: Juraci Paixão Kröhling <[email protected]> Signed-off-by: Juraci Paixão Kröhling <[email protected]> * [pkg/stanza] Move encoding concerns to dedicated package (#26019) * Remove defunct elastic_output reference from operator README (#26034) **Description:** The operators README file still included a reference to the `elastic_output` which was removed a while back in favor of the `elasticsearchexporter`. This PR replaces the example with a file_output example, which still exists. * Translate unit from words to UCUM in the prometheus receiver (#25887) Fixes https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23208 This is the reverse of the logic in the prometheus exporter, which changes UCUM to words. * [chore][fileconsumer] Fix bug where flush period would be forgotten (#26033) Adds a test for and fixes a bug introduced in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/25847 No changelog is necessary because the bug was introduced yesterday and was never released. * [chore][fileconsumer] Extract splitter, flusher, multiline, encoding into dedicated package (#25914) ~~Follows #26019~~ There are several structs in `pkg/stanza/operator/helper` which define and manage tokenization related behaviors. `fileconsumer` has a dependency on these, as do a few other components such as tcp and udp receivers. These structs have complex interrelationships which I find difficult to reason about and difficult to manage. Their current location in the `helpers` package is unhelpful and unnecessary. I'd like to move them into a dedicated package where I can start to untangle and hopefully simplify them. * [chore] [exporter/coralogix] Document and test attributes ordering (#26027) **Description:** We were seeing reports of cases where unexpected attribute has been picked up as subsytem / application attribute. This PR adds some clarifications on how the attribute is picked. Also adds tests to validate this behavior. **Link to tracking Issue:** N/A **Testing:** N/A **Documentation:** Added documentation regarding attributes ordering. --------- Signed-off-by: Matej Gera <[email protected]> * [chore][pkg/stanza] Move tokenization tests into tokenizetest package (#25997) * [chore] Remove non-ucum units from prometheus readme (#26035) Follow-up to https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/24647 Addresses comment in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/25887#pullrequestreview-1588092792 * [chore] dependabot updates Tue Aug 22 20:28:42 UTC 2023 (#26039) Bump cloud.google.com/go/spanner from 1.47.0 to 1.48.0 in /receiver/googlecloudspannerreceiver Bump github.com/ClickHouse/clickhouse-go/v2 from 2.13.0 to 2.13.2 in /exporter/clickhouseexporter Bump github.com/aliyun/aliyun-log-go-sdk from 0.1.51 to 0.1.53 in /exporter/alibabacloudlogserviceexporter Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /exporter/awscloudwatchlogsexporter Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /exporter/awsxrayexporter Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /extension/observer/ecsobserver Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /internal/aws/cwlogs Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /internal/aws/xray Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /internal/metadataproviders Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /processor/resourcedetectionprocessor Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /receiver/awscontainerinsightreceiver Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.327 in /receiver/awsecscontainermetricsreceiver Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.328 in /exporter/awsemfexporter Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.328 in /exporter/datadogexporter Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.328 in /exporter/kafkaexporter Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.328 in /internal/aws/awsutil Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.328 in /internal/aws/k8s Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.328 in /internal/aws/proxy Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.328 in /internal/aws/xray/testdata/sampleapp Bump github.com/aws/aws-sdk-go from 1.44.323 to 1.44.328 in /receiver/awsxrayreceiver Bump github.com/aws/aws-sdk-go-v2 from 1.20.1 to 1.20.3 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2 from 1.20.1 to 1.21.0 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/config from 1.18.33 to 1.18.35 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/config from 1.18.33 to 1.18.36 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.32 to 1.13.34 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.32 to 1.13.35 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/service/kinesis from 1.18.2 to 1.18.5 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/service/sts from 1.21.2 to 1.21.4 in /extension/sigv4authextension Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /cmd/configschema Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /exporter/awscloudwatchlogsexporter Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /exporter/awsemfexporter Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /exporter/googlecloudpubsubexporter Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /exporter/tanzuobservabilityexporter Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /pkg/ottl Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /processor/k8sattributesprocessor Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /processor/tailsamplingprocessor Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /receiver/awsxrayreceiver Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /receiver/k8sclusterreceiver Bump github.com/google/uuid from 1.3.0 to 1.3.1 in /receiver/skywalkingreceiver Bump github.com/mongodb-forks/digest from 1.0.4 to 1.0.5 in /receiver/mongodbatlasreceiver Bump github.com/sijms/go-ora/v2 from 2.7.11 to 2.7.14 in /receiver/sqlqueryreceiver Bump github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common from 1.0.719 to 1.0.727 in /exporter/tencentcloudlogserviceexporter Bump go.opentelemetry.io/collector/receiver from 0.82.0 to 0.83.0 in /receiver/googlecloudpubsubreceiver Bump golang.org/x/vuln from 1.0.0 to 1.0.1 in /internal/tools Bump gonum.org/v1/gonum from 0.13.0 to 0.14.0 in /pkg/stanza Bump gonum.org/v1/gonum from 0.13.0 to 0.14.0 in /receiver/statsdreceiver Bump google.golang.org/api from 0.136.0 to 0.138.0 in /exporter/f5cloudexporter Bump google.golang.org/api from 0.136.0 to 0.138.0 in /exporter/googlecloudpubsubexporter Bump google.golang.org/api from 0.136.0 to 0.138.0 in /receiver/googlecloudpubsubreceiver Bump google.golang.org/api from 0.136.0 to 0.138.0 in /receiver/googlecloudspannerreceiver * [chore] [processor/transform] Add the missing parenthesis in the README example (#26066) One parenthesis is missing in "Comnbine two attributes" example code. * [exporter/loadbalancingexporter] fix k8s service resolver retaining invalid old endpoints (#25061) **Description:** <Describe what has changed.>  fix k8s service resolver retaining invalid old endpoints In the endpoint update event, the step of deleting the old endpoint is missed, so a connection request will still be made to the old endpoint. **Link to tracking Issue:** <Issue number if applicable> #24914 **Testing:** <Describe what testing was performed and which tests were added.> **Documentation:** <Describe the documentation added.> Signed-off-by: Yuan Fang <[email protected]> Co-authored-by: Juraci Paixão Kröhling <[email protected]> * [chore] dependabot updates Wed Aug 23 04:31:08 UTC 2023 (#26067) Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /exporter/awscloudwatchlogsexporter Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /exporter/awsemfexporter Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /exporter/awsxrayexporter Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /exporter/datadogexporter Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /exporter/kafkaexporter Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /extension/observer/ecsobserver Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /internal/aws/awsutil Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /internal/aws/cwlogs Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /internal/aws/k8s Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /internal/aws/proxy Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /internal/aws/xray Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /internal/aws/xray/testdata/sampleapp Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /internal/metadataproviders Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /processor/resourcedetectionprocessor Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /receiver/awscontainerinsightreceiver Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /receiver/awsecscontainermetricsreceiver Bump github.com/aws/aws-sdk-go from 1.44.328 to 1.44.329 in /receiver/awsxrayreceiver Bump github.com/aws/aws-sdk-go-v2/config from 1.18.33 to 1.18.36 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/config from 1.18.35 to 1.18.36 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.32 to 1.13.35 in /exporter/awskinesisexporter Bump github.com/aws/aws-sdk-go-v2/credentials from 1.13.34 to 1.13.35 in /extension/sigv4authextension Bump github.com/aws/aws-sdk-go-v2/service/sts from 1.21.4 to 1.21.5 in /extension/sigv4authextension Bump github.com/sijms/go-ora/v2 from 2.7.14 to 2.7.15 in /receiver/sqlqueryreceiver Bump github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common from 1.0.727 to 1.0.728 in /exporter/tencentcloudlogserviceexporter Bump go.opentelemetry.io/collector/receiver from 0.82.0 to 0.83.0 in /receiver/googlecloudpubsubreceiver * fix e2e telemetrygen pod to enable e2e-test action (#25905) **Description:** before #25889 is released, the telemetrygen image is still not the latest one, so we need to fix e2e telemetrygen pod yaml to enable e2e-test action again. --------- Signed-off-by: Ziqi Zhao <[email protected]> * Add support for Metrics Slices to ottl len converter (#25890) Fixes #25868 **Description:** <Describe what has changed.>  **Link to tracking Issue:** <Issue number if applicable> **Testing:** <Describe what testing was performed and which tests were added.> **Documentation:** <Describe the documentation added.> * Allow log's body to be set in telemetrygen (#26069) This change adds a flag allowing users to set the log's body. It also sets a log body to a default message. Replaces #26031 Signed-off-by: Juraci Paixão Kröhling <[email protected]> * [exporter/tanzuobservability] Deprecate Tanzu Observability (Wavefront) Exporter (#25904) The Tanzu Observability (Wavefront) Proxy has support for ingesting traces and metrics via OTLP, so the vendor specific exporter is no longer needed. This PR adds a deprecation message. **Link to tracking Issue:** #24225 **Documentation:** Readme updated with deprecation notice and link to migration documentation. * [chore] dependabot updates Wed Aug 23 16:09:23 UTC 2023 (#26077) Bump github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common from 1.0.728 to 1.0.729 in /exporter/tencentcloudlogserviceexporter * receiver_creator: update expr with breaking type fn (#26064) Fixing a bug - These changes adopt the latest version of expr and incorporate a breaking built-in `type` function addition by relocating its implementation to `typeOf`. Fixes #26038 * [chore] Update lb exporter and k8sattributes processor with aws distro (#26079) **Description:** ADOT Collector now includes load balancing exporter and k8s attributes processor. This PR Updates medata.yaml and the components associated readme. * [pkg/resourcetotelemetry] enable exhaustive lint (#26075) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> * [chore] enforce validation of codeowners membership (#24638) **Description:** This adds a validation to our codeowners by forcing us to review and manage the codeowners registered with the OpenTelemetry github organization. * [chore] Check component API (#25855) **Description:** Introduce an API checker ensuring components only define a function per component to expose the factory function NewFactory **Testing:** Run locally, run as part of CI * [chore] only check for codeowners on main branch (#26088) Restrict when this step is run to make sure the secret GITHUB_TOKEN is set. * [receiver/memcached] remove duplicate Timeout setting (#26084) Use the Timeout setting from ScraperHelper instead --------- Signed-off-by: Alex Boten <[email protected]> * [receiver/k8sobjects] Switch to current config testing pattern (#26078) **Description:** Updates the component to use the latest config testing patterns. * [pkg/translator/loki] Moved labels names normalization to LogToLokiEntry function (#26093) **Description:** <Describe what has changed.> The public method `LogToLokiEntry` should return normalized label names to release developers who use it from the necessity to normalize them in their code **Testing:** adjusted unit tests of lokitranslator * feat: allow custom port for cassandra connection (#25179) * Add JMX metrics gatherer version 1.29.0-alpha (#26089) Add JMX metrics gatherer version `1.29.0-alpha`. cc @open-telemetry/java-contrib-approvers * [chore] fix exhaustive lint for processor groupbyattrs (#24919) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> Co-authored-by: Alex Boten <[email protected]> * [exporter/awsxray] Feature Gate to Allow fully random Trace IDs in XRay Exporter (#26041) Currently, the AWS XRay Exporter will drop segments/traces with trace IDs where the first 32 bits, when converted to UNIX Epoch time, are not within the past 28 days. This change is to add an Alpha feature gate `exporter.awsxray.skiptimestampvalidation`, which is disabled by default. If enabled, the timestamp restriction is removed so that users do not need to use the AWS XRay ID Generator. - Updated unit test in `config_test.go` - Added unit tests in `factory_test.go` and `segment_test.go` - All other unit tests related to trace ID are passing. * added a new postgresql metric. (#26086) Added new metric `postgresql.time_files` which is number of temp files detected in a database. Fixes #26080 * [receiver/zookeeper] remove duplicate Timeout field (#26082) Rely on the scraper helper's Timeout instead --------- Signed-off-by: Alex Boten <[email protected]> * [chore] [exporter/datasetexporter]: Update dataset-go to 0.14.0 (#26094) Upgrade dataset-go from 0.12.1 to 0.14.0 Next release is happening on 2023-08-28 (Monday) - https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/release.md - but next dependant is running on 2023-08-31 (Wednesday) - https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/.github/dependabot.yml - so it will not get updated on time. * [receiver/azuremonitorreceiver] Added new attributes for metrics (#24774) Added new attributes: name, type and resource group to the metrics * [chore] Add bryan-aguilar as EMF Exporter codeowner (#26100) **Description:** Adding myself as a codeowner of AWS EMF Exporter * [chore] add cmd group target and use it for testing and lint (#25876) Fixes #25875 * Parse scope information from otel_scope_name, otel_scope_version, and otel_scope_info (#25898) **Description:** Implements this specification: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/compatibility/prometheus_and_openmetrics.md#instrumentation-scope, with https://github.com/open-telemetry/opentelemetry-specification/pull/3660 Instrumentation Scope > Each otel_scope_info metric point present in a batch of metrics SHOULD be dropped from the incoming scrape, and converted to an instrumentation scope. The otel_scope_name and otel_scope_version labels, if present, MUST be converted to the Name and Version of the Instrumentation Scope. Additional labels MUST be added as scope attributes, with keys and values unaltered. Other metrics in the batch which have otel_scope_name and otel_scope_version labels that match an instrumentation scope MUST be placed within the matching instrumentation scope, and MUST remove those labels. > Metrics which are not found to be associated with an instrumentation scope MUST all be placed within an empty instrumentation scope, and MUST not have any labels removed. It does this by: * For all metrics, use `otel_scope_name` and `otel_scope_version` to set the scope name and version. * For `otel_scope_info` metrics, use `otel_scope_name` and `otel_scope_version` as the scope name and version, and all other labels (other than `job`/`instance`/`__name__`) as scope attributes. * Change `map[metricName]metricFamily` to `map[scope][metricName]metricFamily` to sort metrics by instrumentation scope as they come in, and to make writing the scope easier on commit. **Link to tracking Issue:** Fixes https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/25870 * [receiver/jaeger] Removing remote sampling configuration code (#24186) Most remote sampling code is already removed from the jaeger receiver via #6633, but the configuration is still there. It is unused and should be removed. The only reason I can think of for retaining it is so that old configuration files don't stop the Collector from running. However, given that this is a no-op, I think it would be better to have an error and let the customers reconfigure their collectors. Here, I'm assuming that configuring a struct which doesn't exist will indeed raise an error. There is no changelog entry since the change is mostly an implementation detail. --------- Co-authored-by: Juraci Paixão Kröhling <[email protected]> * [pkg/pdatatest] enable exhaustive lint (#26105) **Description:** related #23266 Signed-off-by: Ziqi Zhao <[email protected]> * feat: add duration to int float converters (#25069) Description: Allows conversion of durations to nanoseconds, microseconds, milliseconds, seconds, minutes or hours Link to tracking Issue: Related to #24686 Testing: Unit tests for each converter Documentation: --------- Co-authored-by: Tyler Helmuth <12352919+TylerHelm…

mircohacker requested a review from a team January 24, 2023 09:08

mircohacker requested a review from Aneurysm9 as a code owner January 24, 2023 09:08

github-actions bot assigned MovieStoreGuy Jan 24, 2023

github-actions bot added the exporter/awsemf awsemf exporter label Jan 24, 2023

mircohacker force-pushed the retainInitialValueOfDeltaMetric branch from 01de021 to 4bbd1f6 Compare January 24, 2023 09:15

MovieStoreGuy added the ready to merge Code review completed; ready to merge by maintainers label Feb 8, 2023

mircohacker force-pushed the retainInitialValueOfDeltaMetric branch from 4bbd1f6 to e8d3070 Compare February 8, 2023 19:36

Aneurysm9 approved these changes Feb 9, 2023

View reviewed changes

TylerHelmuth reviewed Feb 10, 2023

View reviewed changes

mircohacker force-pushed the retainInitialValueOfDeltaMetric branch from e8d3070 to 511315f Compare February 11, 2023 19:16

mircohacker force-pushed the retainInitialValueOfDeltaMetric branch from 511315f to 4b71292 Compare February 24, 2023 13:31

codeboten reviewed Feb 27, 2023

View reviewed changes

mircohacker force-pushed the retainInitialValueOfDeltaMetric branch from 4b71292 to d60aa15 Compare February 28, 2023 09:31

[exporter/awsemfexporter] Add config option to retain the initial val…

bf67b13

…ue of delta metrics (open-telemetry#16218)

mircohacker force-pushed the retainInitialValueOfDeltaMetric branch from d60aa15 to bf67b13 Compare March 1, 2023 20:22

codeboten approved these changes Mar 1, 2023

View reviewed changes

codeboten merged commit a47c243 into open-telemetry:main Mar 1, 2023

mircohacker deleted the retainInitialValueOfDeltaMetric branch March 6, 2023 08:34

This was referenced Mar 6, 2023

[exporter/awsemf] Exporter ignores the first batch of a metric send to cloudwatch #16218

Closed

[exporter/awsemf] Exporter ignores the first batch of a metric send to cloudwatch aws-observability/aws-otel-collector#1653

Closed

mizzzto mentioned this pull request Apr 25, 2023

[exporter/awsemf] Exporter still ignores the first batch of a metric send to CloudWatch aws-observability/aws-otel-collector#1991

Closed

jameshi16 mentioned this pull request Jul 10, 2023

[exporter/awsemfexporter] Propagate RetainInitialValueOfDeltaMetric to translateOTelToGroupedMetric #24051

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[exporter/awsemfexporter] Add config option to retain the initial value of delta metrics (#16218) #17988

[exporter/awsemfexporter] Add config option to retain the initial value of delta metrics (#16218) #17988

mircohacker commented Jan 24, 2023

linux-foundation-easycla bot commented Jan 24, 2023 •

edited

Loading

runforesight bot commented Jan 26, 2023 •

edited

Loading

✅ tracegen workflow has finished in 1 minute 3 seconds (1 minute 22 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

⭕ build-and-test-windows workflow has finished in 8 seconds (43 minutes 7 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ check-links workflow has finished in 1 minute 37 seconds (57 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ telemetrygen workflow has finished in 1 minute 33 seconds (1 minute 39 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ changelog workflow has finished in 2 minutes 31 seconds and finished at 1st Mar, 2023.

build-and-test workflow has finished in 15 minutes 3 seconds (57 minutes 4 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 28 seconds (5 minutes 44 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ load-tests workflow has finished in 7 minutes 1 second (10 minutes 39 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ e2e-tests workflow has finished in 11 minutes 47 seconds (4 minutes 54 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

MovieStoreGuy commented Jan 27, 2023

MovieStoreGuy commented Feb 6, 2023

Aneurysm9 commented Feb 7, 2023

MovieStoreGuy commented Feb 8, 2023

mircohacker commented Feb 8, 2023

MovieStoreGuy commented Feb 8, 2023

TylerHelmuth left a comment

mircohacker commented Feb 11, 2023

MovieStoreGuy commented Feb 21, 2023

mircohacker commented Feb 24, 2023

codeboten left a comment

mircohacker commented Feb 28, 2023

codeboten commented Feb 28, 2023

mircohacker commented Mar 1, 2023

[exporter/awsemfexporter] Add config option to retain the initial value of delta metrics (#16218) #17988

[exporter/awsemfexporter] Add config option to retain the initial value of delta metrics (#16218) #17988

Conversation

mircohacker commented Jan 24, 2023

linux-foundation-easycla bot commented Jan 24, 2023 • edited Loading

runforesight bot commented Jan 26, 2023 • edited Loading

Foresight Summary

✅ tracegen workflow has finished in 1 minute 3 seconds (1 minute 22 seconds less than main branch avg.) and finished at 26th Jan, 2023.

⭕ build-and-test-windows workflow has finished in 8 seconds (43 minutes 7 seconds less than main branch avg.) and finished at 1st Mar, 2023.

✅ check-links workflow has finished in 1 minute 37 seconds (57 seconds less than main branch avg.) and finished at 1st Mar, 2023.

✅ telemetrygen workflow has finished in 1 minute 33 seconds (1 minute 39 seconds less than main branch avg.) and finished at 1st Mar, 2023.

✅ changelog workflow has finished in 2 minutes 31 seconds and finished at 1st Mar, 2023.

build-and-test workflow has finished in 15 minutes 3 seconds (57 minutes 4 seconds less than main branch avg.) and finished at 1st Mar, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 28 seconds (5 minutes 44 seconds less than main branch avg.) and finished at 1st Mar, 2023.

✅ load-tests workflow has finished in 7 minutes 1 second (10 minutes 39 seconds less than main branch avg.) and finished at 1st Mar, 2023.

✅ e2e-tests workflow has finished in 11 minutes 47 seconds (4 minutes 54 seconds less than main branch avg.) and finished at 1st Mar, 2023.

MovieStoreGuy commented Jan 27, 2023

MovieStoreGuy commented Feb 6, 2023

Aneurysm9 commented Feb 7, 2023

MovieStoreGuy commented Feb 8, 2023

mircohacker commented Feb 8, 2023

MovieStoreGuy commented Feb 8, 2023

TylerHelmuth left a comment

Choose a reason for hiding this comment

mircohacker commented Feb 11, 2023

MovieStoreGuy commented Feb 21, 2023

mircohacker commented Feb 24, 2023

codeboten left a comment

Choose a reason for hiding this comment

mircohacker commented Feb 28, 2023

codeboten commented Feb 28, 2023

mircohacker commented Mar 1, 2023

linux-foundation-easycla bot commented Jan 24, 2023 •

edited

Loading

runforesight bot commented Jan 26, 2023 •

edited

Loading

✅ tracegen workflow has finished in 1 minute 3 seconds (1 minute 22 seconds less than `main` branch avg.) and finished at 26th Jan, 2023.

⭕ build-and-test-windows workflow has finished in 8 seconds (43 minutes 7 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ check-links workflow has finished in 1 minute 37 seconds (57 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ telemetrygen workflow has finished in 1 minute 33 seconds (1 minute 39 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

build-and-test workflow has finished in 15 minutes 3 seconds (57 minutes 4 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ prometheus-compliance-tests workflow has finished in 3 minutes 28 seconds (5 minutes 44 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ load-tests workflow has finished in 7 minutes 1 second (10 minutes 39 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.

✅ e2e-tests workflow has finished in 11 minutes 47 seconds (4 minutes 54 seconds less than `main` branch avg.) and finished at 1st Mar, 2023.