-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[sumologicexporter]: replace with Sumo Logic Exporter from Sumo Logic Distribution for OpenTelemetry #32315
Closed
sumo-drosiek
wants to merge
6
commits into
open-telemetry:main
from
sumo-drosiek:drosiek-move-exporter
Closed
[sumologicexporter]: replace with Sumo Logic Exporter from Sumo Logic Distribution for OpenTelemetry #32315
sumo-drosiek
wants to merge
6
commits into
open-telemetry:main
from
sumo-drosiek:drosiek-move-exporter
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Dominik Rosiek <[email protected]>
Signed-off-by: Dominik Rosiek <[email protected]>
Signed-off-by: Dominik Rosiek <[email protected]>
Signed-off-by: Dominik Rosiek <[email protected]>
Signed-off-by: Dominik Rosiek <[email protected]>
Signed-off-by: Dominik Rosiek <[email protected]>
This was referenced Apr 11, 2024
andrzej-stencel
pushed a commit
that referenced
this pull request
Apr 16, 2024
…stograms` functions (#32316) **Description:** Adds files from Sumo Logic Distribution for OpenTelemetry which are new in order to simplify review and make simpler to move the whole component. This PR is made to simplify #32315 **Link to tracking Issue:** #31479 **Testing:** Unit Tests, also this code is use by our customers and internally. It is not used by now by the component, but will be when migration will be finished **Documentation:** N/A Signed-off-by: Dominik Rosiek <[email protected]>
andrzej-stencel
pushed a commit
that referenced
this pull request
Apr 17, 2024
… obtain endpoint (#32383) **Description:** Use Sumo Logic Extension for authentication and to obtain the endpoint. This is part of #31479 to make #32315 smaller, therefore it may have some unused code (like `getSignalURL`). Also OTLP format is not supported yet for neither of the formats It also partially fixes #32375 (partially because as long as there is no a otlp support, data won't be pushed to Sumo Logic) **Link to tracking Issue:** #31479 **Testing:** - unit tests - manual tests **Documentation:** - ensures that Sumo Logic Extension documentation is valid --------- Signed-off-by: Dominik Rosiek <[email protected]>
This was referenced Apr 22, 2024
mx-psi
pushed a commit
that referenced
this pull request
Apr 23, 2024
…32595) **Description:** Use new formatter for prometheus, which fixes metric name sanitization. This also refactors the formatter itself, which is part of #31479 and reduces size of #32315 **Link to tracking Issue:** Applies SumoLogic/sumologic-otel-collector#211 and more, but this one is a bug fix **Testing:** Unit tests **Documentation:** N/A --------- Signed-off-by: Dominik Rosiek <[email protected]>
mx-psi
pushed a commit
that referenced
this pull request
Apr 23, 2024
**Description:** Refactor sender in order to prepare for adding OTLP formats Reduces size of #32315 **Link to tracking Issue:** #31479 **Testing:** Minimal changes to unit tests to ensure that everything works like before **Documentation:** N/A --------- Signed-off-by: Dominik Rosiek <[email protected]>
mx-psi
added a commit
that referenced
this pull request
Apr 24, 2024
**Description:** Refactor sender in order to prepare for adding OTLP formats Reduces size of #32315 Reopens #32592 due to revert due to test issues **Link to tracking Issue:** #31479 **Testing:** Minimal changes to unit tests to ensure that everything works like before **Documentation:** N/A --------- Signed-off-by: Dominik Rosiek <[email protected]> Co-authored-by: Pablo Baeyens <[email protected]>
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
andrzej-stencel
pushed a commit
that referenced
this pull request
May 8, 2024
**Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> * remove suppport for carbon2 and graphite * add support for otlp format * do not support metadata attributes * do not support source headers * set otlp as default format This PR reduces size of #32315 **Link to tracking Issue:** #31479 **Testing:** - unit tests - manual tests **Documentation:** - Readme --------- Signed-off-by: Dominik Rosiek <[email protected]> Co-authored-by: Adam Boguszewski <[email protected]>
Closed as inactive. Feel free to reopen if this PR is still being worked on. |
andrzej-stencel
pushed a commit
that referenced
this pull request
May 13, 2024
**Description:** * set OTLP as default format * add support for OTLP format * do not support metadata attributes * do not support source headers **Link to tracking Issue:** #32315 **Testing:** * unit tests **Documentation:** * inline comments * readme --------- Signed-off-by: Dominik Rosiek <[email protected]>
This was referenced May 13, 2024
mx-psi
pushed a commit
that referenced
this pull request
May 13, 2024
**Description:** Adds support for sticky session in order to better support AWS LB. This code is moved from Sumo Logic repository **Link to tracking Issue:** #32315 **Testing:** Tested manually **Documentation:** N/A --------- Signed-off-by: Dominik Rosiek <[email protected]>
mx-psi
pushed a commit
that referenced
this pull request
May 14, 2024
**Description:** remove deprecated configuration options and remove unused code Depends on #33011 **Link to tracking Issue:** #32315 **Testing:** * Unit tests **Documentation:** * Update Readme --------- Signed-off-by: Dominik Rosiek <[email protected]>
andrzej-stencel
pushed a commit
that referenced
this pull request
May 22, 2024
**Description:** adds support for tracing It should be the last PR which moves code from Sumo Logic repository. I will compare the codebase after merging it **Link to tracking Issue:** #32315 **Testing:** * unit tests **Documentation:** * Readme --------- Signed-off-by: Dominik Rosiek <[email protected]>
steves-canva
added a commit
to Canva/opentelemetry-collector-contrib
that referenced
this pull request
Jun 14, 2024
* Update module google.golang.org/api to v0.178.0 (#32927) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google.golang.org/api](https://togithub.com/googleapis/google-api-go-client) | `v0.177.0` -> `v0.178.0` | [![age](https://developer.mend.io/api/mc/badges/age/go/google.golang.org%2fapi/v0.178.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/google.golang.org%2fapi/v0.178.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/google.golang.org%2fapi/v0.177.0/v0.178.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/google.golang.org%2fapi/v0.177.0/v0.178.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>googleapis/google-api-go-client (google.golang.org/api)</summary> ### [`v0.178.0`](https://togithub.com/googleapis/google-api-go-client/releases/tag/v0.178.0) [Compare Source](https://togithub.com/googleapis/google-api-go-client/compare/v0.177.0...v0.178.0) ##### Features - **all:** Auto-regenerate discovery clients ([#​2561](https://togithub.com/googleapis/google-api-go-client/issues/2561)) ([2d22d11](https://togithub.com/googleapis/google-api-go-client/commit/2d22d11df9643a4fad0f9e952d7a92a419370122)) - **all:** Auto-regenerate discovery clients ([#​2564](https://togithub.com/googleapis/google-api-go-client/issues/2564)) ([b313e4b](https://togithub.com/googleapis/google-api-go-client/commit/b313e4bd70e601fd7a2a931f168fb1dece980e75)) - **all:** Auto-regenerate discovery clients ([#​2565](https://togithub.com/googleapis/google-api-go-client/issues/2565)) ([0843d21](https://togithub.com/googleapis/google-api-go-client/commit/0843d217048b2e713c0d273b95b33afb99926a8c)) - **all:** Auto-regenerate discovery clients ([#​2567](https://togithub.com/googleapis/google-api-go-client/issues/2567)) ([76b27f1](https://togithub.com/googleapis/google-api-go-client/commit/76b27f162032649ddb3cb3f06ed24c7333b3fa66)) - **all:** Auto-regenerate discovery clients ([#​2568](https://togithub.com/googleapis/google-api-go-client/issues/2568)) ([d922e3b](https://togithub.com/googleapis/google-api-go-client/commit/d922e3b559ce5832941390b4f9bf91210e3f6579)) - **all:** Auto-regenerate discovery clients ([#​2570](https://togithub.com/googleapis/google-api-go-client/issues/2570)) ([f2da582](https://togithub.com/googleapis/google-api-go-client/commit/f2da582c9f6aab240d44c8ebd2dcc43f5096f896)) - **all:** Auto-regenerate discovery clients ([#​2571](https://togithub.com/googleapis/google-api-go-client/issues/2571)) ([0c976dc](https://togithub.com/googleapis/google-api-go-client/commit/0c976dcc8d1d653f2284ce273493e6714a6d4b2a)) - **gen:** Add internaloption.EnableNewAuthLibrary ([#​2519](https://togithub.com/googleapis/google-api-go-client/issues/2519)) ([8c74bb8](https://togithub.com/googleapis/google-api-go-client/commit/8c74bb83e2bc27188154c506e63a3e0f3a042f55)) - **google-api-go-client:** Add x-goog-api-version header ([#​2563](https://togithub.com/googleapis/google-api-go-client/issues/2563)) ([fe54ffd](https://togithub.com/googleapis/google-api-go-client/commit/fe54ffd92359506fca1ffd70dc647db0ab9a903c)) ##### Documentation - Update commit style in CONTRIBUTING ([#​2566](https://togithub.com/googleapis/google-api-go-client/issues/2566)) ([5e44215](https://togithub.com/googleapis/google-api-go-client/commit/5e44215df618fcafd5f6c1bbe259062cddd32f1a)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNDAuMTAiLCJ1cGRhdGVkSW5WZXIiOiIzNy4zNDAuMTAiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbImRlcGVuZGVuY2llcyIsInJlbm92YXRlYm90Il19--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * chore: add constants for prometheus translation (#32830) **Description:** This is a cleanup to consolidate the constants used for prometheus translation in a single place. --------- Co-authored-by: Curtis Robert <[email protected]> * sumologicexporter!: change metrics behavior (#32737) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> * remove suppport for carbon2 and graphite * add support for otlp format * do not support metadata attributes * do not support source headers * set otlp as default format This PR reduces size of #32315 **Link to tracking Issue:** #31479 **Testing:** - unit tests - manual tests **Documentation:** - Readme --------- Signed-off-by: Dominik Rosiek <[email protected]> Co-authored-by: Adam Boguszewski <[email protected]> * Update module github.com/aws/aws-sdk-go to v1.52.4 (#32928) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/aws/aws-sdk-go](https://togithub.com/aws/aws-sdk-go) | `v1.52.3` -> `v1.52.4` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2faws%2faws-sdk-go/v1.52.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2faws%2faws-sdk-go/v1.52.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2faws%2faws-sdk-go/v1.52.3/v1.52.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2faws%2faws-sdk-go/v1.52.3/v1.52.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>aws/aws-sdk-go (github.com/aws/aws-sdk-go)</summary> ### [`v1.52.4`](https://togithub.com/aws/aws-sdk-go/blob/HEAD/CHANGELOG.md#Release-v1524-2024-05-07) [Compare Source](https://togithub.com/aws/aws-sdk-go/compare/v1.52.3...v1.52.4) \=== ##### Service Client Updates - `service/b2bi`: Updates service documentation - `service/budgets`: Updates service API and documentation - This release adds tag support for budgets and budget actions. - `service/resiliencehub`: Updates service API, documentation, and paginators - `service/route53profiles`: Updates service API and documentation </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNDAuMTAiLCJ1cGRhdGVkSW5WZXIiOiIzNy4zNDAuMTAiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbImRlcGVuZGVuY2llcyIsInJlbm92YXRlYm90Il19--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * [chore][receiver/sqlserver] Update documentation for darwin and linux OS (#32878) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> With the enhancement in https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/30297, the SQL Server receiver can now run on MacOS and Linux, since it no longer solely relies on Windows. The difference in functionality has (hopefully) been documented clearly between the different platforms, so the unsupported warning can be removed. I missed this in my previous PRs. * Update module github.com/grafana/loki/pkg/push to v0.0.0-20240507085123-772616cd8f5c (#32897) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/grafana/loki/pkg/push](https://togithub.com/grafana/loki) | `v0.0.0-20240506154431-a772ed705c65` -> `v0.0.0-20240507085123-772616cd8f5c` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240507085123-772616cd8f5c?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240507085123-772616cd8f5c?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240506154431-a772ed705c65/v0.0.0-20240507085123-772616cd8f5c?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240506154431-a772ed705c65/v0.0.0-20240507085123-772616cd8f5c?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNDAuMTAiLCJ1cGRhdGVkSW5WZXIiOiIzNy4zNDAuMTAiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbImRlcGVuZGVuY2llcyIsInJlbm92YXRlYm90Il19--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * chore(deps): update dependency go to v1.22.3 (#32621) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [go](https://go.dev/) ([source](https://togithub.com/golang/go)) | toolchain | minor | `1.21.9` -> `1.22.3` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>golang/go (go)</summary> ### [`v1.22.3`](https://togithub.com/golang/go/compare/go1.22.2...go1.22.3) ### [`v1.22.2`](https://togithub.com/golang/go/compare/go1.22.1...go1.22.2) ### [`v1.22.1`](https://togithub.com/golang/go/compare/go1.22.0...go1.22.1) ### [`v1.22.0`](https://togithub.com/golang/go/compare/go1.21.7...go1.22rc1) ### [`v1.21.10`](https://togithub.com/golang/go/compare/go1.21.9...go1.21.10) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zMTMuMSIsInVwZGF0ZWRJblZlciI6IjM3LjM0MC4xMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIiwicmVub3ZhdGVib3QiXX0=--> --------- Signed-off-by: Juraci Paixão Kröhling <[email protected]> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> Co-authored-by: Juraci Paixão Kröhling <[email protected]> * [receiver/haproxy] Unit test timing out (#32940) I wasn't able to reproduce the exact situation from CI, but the tests I changed weren't completing at all here. Looking closely, it looks like they never closed the stream, something the first test does. After closing it, the tests start passing locally for me. While I'm not confident this will fix the CI flaky failures, this does make the test work locally for me. Fixes #32877 Signed-off-by: Juraci Paixão Kröhling <[email protected]> Signed-off-by: Juraci Paixão Kröhling <[email protected]> Co-authored-by: Alex Boten <[email protected]> * [connector/servicegraph] Remove use of host.GetExporters (#32902) Fixes #31628 Signed-off-by: Juraci Paixão Kröhling <[email protected]> --------- Signed-off-by: Juraci Paixão Kröhling <[email protected]> Co-authored-by: Alex Boten <[email protected]> * [exporter/loadbalancing] Improve the performance when merging traces belonging to the same backend (#32032) **Description:** no need to reimplement that in an extremely allocation-inefficient fashion. I'm actually not sure why mergeTraces() and mergeMetrics() need to exist in the first place; all the other exporters coupled with the batch processor work just fine, not sure why loadbalancing would be special. https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30141 seems to imply they were implemented to improve performance, but I don't really understand why batch processor would not have been sufficient for that improvement originally. benchmarks before: ``` goos: darwin goarch: arm64 pkg: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter BenchmarkMergeTraces_X100-8 50214 23507 ns/op BenchmarkMergeTraces_X500-8 10000 113952 ns/op BenchmarkMergeTraces_X1000-8 5208 226062 ns/op BenchmarkMergeMetrics_X100-8 64933 18540 ns/op BenchmarkMergeMetrics_X500-8 12885 91418 ns/op BenchmarkMergeMetrics_X1000-8 6590 184584 ns/op PASS ok github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter 9.783s ``` and after: ``` goos: darwin goarch: arm64 pkg: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter BenchmarkMergeTraces_X100-8 295886529 3.836 ns/op BenchmarkMergeTraces_X500-8 309865370 3.833 ns/op BenchmarkMergeTraces_X1000-8 310739948 3.800 ns/op BenchmarkMergeMetrics_X100-8 315567813 3.841 ns/op BenchmarkMergeMetrics_X500-8 310341650 3.849 ns/op BenchmarkMergeMetrics_X1000-8 314292003 3.830 ns/op PASS ok github.com/open-telemetry/opentelemetry-collector-contrib/exporter/loadbalancingexporter 10.733s ``` **Link to tracking Issue:** n/a **Testing:** unit tests pass & cpu time for our collectors using loadbalancingexporter (12 replicas, total of 25k-40k spans/sec) went from 800ms-1400ms/sec down to <40msec/sec. **Documentation:** none --------- Co-authored-by: Juraci Paixão Kröhling <[email protected]> * Move Aneurysm9 to emeritus status (#32943) I have been unable to provide this position the bandwidth that it deserves and it is time to formalize recognition of that fact. Signed-off-by: Anthony J Mirabella <[email protected]> * [receiver/googlecloudpubsubreceiver] Fix memory leak during shutdown (#32361) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This PR contains the following changes: 1. Add `Close` call to the receiver's GRPC client. Without this, goroutines were being leaked on shutdown. 2. Change `grpc.Dial` -> `grpc.NewClient`. They offer the same functionality, but `Dial` is being deprecated in favor of `NewClient`. 3. Enable `goleak` checks on this receiver to help ensure no goroutines are being leaked. 4. Change a couple `Assert.Nil` calls to `Assert.NoError`. The output of `NoError` includes the error message if hit, `Nil` simply includes the object's address, i.e. `&status.Error{s:(*status.Status)(0xc00007e158)}` **Link to tracking Issue:** <Issue number if applicable> #30438 **Testing:** <Describe what testing was performed and which tests were added.> All existing tests are passing, as well as added goleak check. * [chore] Add some docs to readme regarding file exporter (#32855) Admittedly I tested this via docker-compose. I can expand to having a full docker-compose file with telemetry gen if that's preferred. --------- Co-authored-by: Alex Boten <[email protected]> * [receiver/vcenter] Fixes Cluster Resource Attributes for Datastore Resource (#32687) **Description:** <Describe what has changed.> Removed the `vcenter.cluster.name` resource attribute from all Datastore resources. **Link to tracking Issue:** <Issue number if applicable> #32674 **Testing:** <Describe what testing was performed and which tests were added.> Unit/integration tests updated and tested. Local environment tested. **Documentation:** <Describe the documentation added.> New documentation generated based on the metadata. --------- Co-authored-by: Daniel Jaglowski <[email protected]> Co-authored-by: Curtis Robert <[email protected]> * [receiver/vcenter] Adds New Packet Dropped Rate Metric for VMs (#32930) **Description:** <Describe what has changed.> Adds new default disabled (with Warning log for default enabled on next release) metric `vcenter.vm.network.packet.drop.rate` for Virtual Machines. This metric makes use of the `droppedRx` and `droppedTx` Network performance metrics detailed [here](https://vdc-repo.vmware.com/vmwb-repository/dcr-public/d1902b0e-d479-46bf-8ac9-cee0e31e8ec0/07ce8dbd-db48-4261-9b8f-c6d3ad8ba472/network_counters.html) for Virtual machines. This would use the same metric attributes as the other VM packet metrics and closely match `vcenter.vm.network.packet.rate` in every other way. **Link to tracking Issue:** <Issue number if applicable> #32929 **Testing:** <Describe what testing was performed and which tests were added.> Unit/integration tests updated and tested. Local environment tested. **Documentation:** <Describe the documentation added.> New documentation generated based on the metadata. * fix(deps): update module github.com/open-telemetry/otel-arrow to v0.22.0 (#32105) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/open-telemetry/otel-arrow](https://togithub.com/open-telemetry/otel-arrow) | `v0.18.0` -> `v0.22.0` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fopen-telemetry%2fotel-arrow/v0.22.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2fopen-telemetry%2fotel-arrow/v0.22.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2fopen-telemetry%2fotel-arrow/v0.18.0/v0.22.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fopen-telemetry%2fotel-arrow/v0.18.0/v0.22.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>open-telemetry/otel-arrow (github.com/open-telemetry/otel-arrow)</summary> ### [`v0.22.0`](https://togithub.com/open-telemetry/otel-arrow/releases/tag/v0.22.0) [Compare Source](https://togithub.com/open-telemetry/otel-arrow/compare/v0.21.0...v0.22.0) Includes [#​178](https://togithub.com/open-telemetry/otel-arrow/issues/178). [CHANGELOG](https://togithub.com/open-telemetry/otel-arrow/blob/main/CHANGELOG.md) ### [`v0.21.0`](https://togithub.com/open-telemetry/otel-arrow/releases/tag/v0.21.0) [Compare Source](https://togithub.com/open-telemetry/otel-arrow/compare/v0.20.0...v0.21.0) See the [CHANGELOG](https://togithub.com/open-telemetry/otel-arrow/blob/main/CHANGELOG.md). ### [`v0.20.0`](https://togithub.com/open-telemetry/otel-arrow/releases/tag/v0.20.0) [Compare Source](https://togithub.com/open-telemetry/otel-arrow/compare/v0.19.0...v0.20.0) ##### What's Changed - Backport lint fixes from OTel-Collector-Contrib PR 31996 by [@​jmacd](https://togithub.com/jmacd) in [https://github.com/open-telemetry/otel-arrow/pull/163](https://togithub.com/open-telemetry/otel-arrow/pull/163) - Upgrade collector to v0.97.0 by [@​moh-osman3](https://togithub.com/moh-osman3) in [https://github.com/open-telemetry/otel-arrow/pull/164](https://togithub.com/open-telemetry/otel-arrow/pull/164) **Full Changelog**: https://github.com/open-telemetry/otel-arrow/compare/v0.19.0...v0.20.0 ### [`v0.19.0`](https://togithub.com/open-telemetry/otel-arrow/releases/tag/v0.19.0) [Compare Source](https://togithub.com/open-telemetry/otel-arrow/compare/v0.18.0...v0.19.0) See [CHANGELOG.md](https://togithub.com/open-telemetry/otel-arrow/blob/main/CHANGELOG.md#0190---2024-03-26) for release notes. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjM0MC4xMCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * [remotetapprocessor] use 'time/rate' to limit traffic (#32481) bug: The remotetapprocessor `limit` configure doesn't work. how to fix: use `time/rate` to limit traffic. Resolves https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32385 --------- Co-authored-by: Andrzej Stencel <[email protected]> * [receiver/vcenter] Switches Over Metadata Configs Waiting for v0.100.0 Release (#32913) **Description:** <Describe what has changed.> A number of configurations were disabled by default and had warnings that they were going to be enabled in v0.101.0 (1 metric had a warning that it was going to be removed). Now that v0.100.0 has been release, I have removed all of these warnings, and made the modifications that the warnings "warned" about. I have also updated the tests to reflect this. **Link to tracking Issue:** <Issue number if applicable> #32803 #32805 #32821 #32531 #32557 **Testing:** <Describe what testing was performed and which tests were added.> Unit/integration tests updated and tested. Local environment tested. **Documentation:** <Describe the documentation added.> New documentation generated based on the metadata. * [chore][fileconsumer] Skip flaky TestFlushPeriodEOF on windows (#32946) See https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32715 This also adds a bit more debugging info for other tests which fail on the same expectation, since it's not very obvious what was expected vs actually found. * [chore][CI/CD][arm] Trigger arm runs on label (#32955) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> I found in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/32948 that the label `Run ARM` has been added, but the `build-and-test-arm / arm-unittest-matrix (pull_request) ` workflow is still skipped. This is because the `label` action does not trigger a retry. From [documentation](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request): ``` if no activity types are specified, the workflow runs when a pull request is opened or reopened or when the head branch of the pull request is updated. ``` We need to specify that labelling issues should trigger the workflow to check to see if it needs to run again. I've copied the added section from the Windows workflow. I also added that we should only run on PRs against `main`. **Testing:** This PR shows it's working as it should now. Arm test was [originally skipped](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/9009218559/job/24753003216?pr=32955), but after adding the label, tests [have started](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/9009223570/job/24753017935?pr=32955) * [chore][receiver/splunkenterprise] Add header to README (#32956) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> The readme for the Splunk Enterprise receiver does not currently have the autogenerated header. This was missing because `mdatagen` requires `<!-- status autogenerated section -->` and `<!-- end autogenerated section -->` to know where to insert the generated data. * [exporter/elasticsearch] Replace go-elasticsearch BulkIndexer with go-docappender (#32359) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> Replace go-elasticsearch BulkIndexer with go-docappender BulkIndexer for Flush function in preparation for reliability fixes. Maintain similar interface and implementation to go-elasticsearch BulkIndexer. Further changes to expose individual `docappender.BulkIndexer` instances are needed down the road but it is out of the scope of this PR. Implications of this change: - flush timeout is now enforced on client side - oversize payload special handling is now removed - go-docappender uses bulk request filterPath which means bulk response is smaller, less JSON parsing and lower CPU usage - document level retry debug logging is removed as retries are done transparently ~~Blocked by #32585~~ **Link to tracking Issue:** <Issue number if applicable> Fixes #32378 **Testing:** Integration test is passing --------- Co-authored-by: Vishal Raj <[email protected]> * fix(test): Skip flaky test around forcing collector re-registration until the root cause is confirmed (#32937) **Description:** Remove flaky test around forcing collector re-registration until the root cause is confirmed **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32785 **Testing:** Unit tests * [chore][receiver/splunkenterprise] Splunkent wire component (#32795) **Description:** Graduate splunkenterprise receiver component to alpha **Link to tracking Issue:** **Testing:** Performed `make otelcontribcol` and ran resulting binary with the following config: ```yaml extensions: basicauth/indexer: client_auth: username: admin password: securityFirst basicauth/cluster_master: client_auth: username: admin password: securityFirst receivers: splunkenterprise: indexer: auth: authenticator: basicauth/indexer endpoint: "https://localhost:8089/" timeout: 45s cluster_master: auth: authenticator: basicauth/cluster_master endpoint: "https://localhost:8089/" timeout: 45s exporters: otlp: endpoint: 127.0.0.1:8000 service: extensions: [basicauth/indexer, basicauth/cluster_master] pipelines: metrics: receivers: [splunkenterprise] exporters: [otlp] ``` and received the following output: ``` sh ~> ./otelcontribcol_linux_amd64 --config=file:config.yaml 2024-05-08T17:34:33.032-0500 info [email protected]/service.go:102 Setting up own telemetry... 2024-05-08T17:34:33.032-0500 info [email protected]/telemetry.go:103 Serving metrics {"address": ":8888", "level": "Normal"} 2024-05-08T17:34:33.032-0500 info [email protected]/receiver.go:310 Development component. May change in the future. {"kind": "receiver", "name": "splunkenterprise", "data_type": "metrics"} 2024-05-08T17:34:33.033-0500 info [email protected]/service.go:169 Starting otelcontribcol... {"Version": "0.100.0-dev", "NumCPU": 16} 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:34 Starting extensions... 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:37 Extension is starting... {"kind": "extension", "name": "basicauth/cluster_master"} 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:52 Extension started. {"kind": "extension", "name": "basicauth/cluster_master"} 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:37 Extension is starting... {"kind": "extension", "name": "basicauth/indexer"} 2024-05-08T17:34:33.033-0500 info extensions/extensions.go:52 Extension started. {"kind": "extension", "name": "basicauth/indexer"} 2024-05-08T17:34:33.033-0500 info [email protected]/service.go:195 Everything is ready. Begin running and processing data. ``` indicating that the collector was able to successfully start with the component configured. **Documentation:** Documentation was updated to indicate change in status from development to alpha --------- Co-authored-by: Curtis Robert <[email protected]> * [pkg/ottl] Added support for timezone in Time converter (#32479) **Description:** Added support for default timezone in Time converter. Timezone is optional and can be specified as so: `Time("2023-05-26 12:34:56", "%Y-%m-%d %H:%M:%S", "America/New_York")` **Link to tracking Issue:** #32140 **Testing:** Unit tests added **Documentation:** Documentation in ottl/Readme updated --------- Co-authored-by: Tyler Helmuth <[email protected]> Co-authored-by: Evan Bradley <[email protected]> * [processor/transform] Add common where clause (#31491) **Description:** Add global conditions with where clause <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> **Link to tracking Issue:** Fixes #27830 **Testing:** Unit tests **Documentation:** TODO ~~The main objective is to extend the `ContextStatements` struct by adding a new `Conditions` parameter. By introducing `Conditions` to `ContextStatements`, we can now apply a global condition to all related statements in `WithStatementSequenceGlobalConditions` function.~~ Thanks in advance for your feedback! If this changes will be fine, I will add common where clause into another context `span`, `metrics`. * Instantiate ID in pkg/stanza/adapter tests (#32966) **Description:** <Describe what has changed.> In https://github.com/open-telemetry/opentelemetry-collector/pull/10069, I am making Type an interface. This means the zero value of Type will be nil - which will cause this test to fail. Initializing ID instead of relying on the zero value fixes this --------- Co-authored-by: Pablo Baeyens <[email protected]> Co-authored-by: Bogdan Drutu <[email protected]> * Add connector usage to the testbed (#32881) **Description:** Added a new component to the testbed called DataConnectors which allows connectors to be added to the testbed config. Also, added a sample connector correctness test using the routingconnector as an example of the usage. **Link to tracking Issue:** #30165 **Testing:** Sample correctness test using routingconnector. **Documentation:** Will update the testbed README with new addition. --------- Co-authored-by: Bryan Aguilar <[email protected]> * [chore][CONTRIBUTING.md] Update adding component directions (#32957) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This is two main changes: 1. Remove `goleak` section. It's now added by default by mdatagen, there's nothing required of users here. 2. Add information and reformat the `Last steps` section of adding a new component. - Move the directions to be bullet points - Add `make generate` to make sure the component's README is updated properly - Explicitly point out that stability and distribution needs to be updated in `metadata.yaml` - Add step for adding the component to the releases repo. My understanding is that if a component is `Alpha`, it should be included in the release, so I've made that a noted part of the steps here. (Let me know if we should word this in a more "optional" way.) --------- Co-authored-by: Pablo Baeyens <[email protected]> * [opampextension]: Move custom message interface to separate module (#32951) **Description:** <Describe what has changed.> * Breaks our the custom message interface to a separate module, so other components can use the interface without needing to import the `opampextension` module in its entirety. We could temporarily alias the old methods if we'd like, but I think that the CustomMessage stuff has been so short lived that, in addition to the alpha status of the opampextension component, it feels justified to just skip the deprecation process and move it to a new module. **Link to tracking Issue:** Closes #32950 **Testing:** * Covered by existing unit tests **Documentation:** * Added more documentation on usage in the new module. * Modified opampextension docs to point to the new module. * [chore] Add contrib confmap providers to otelcontribcol (#32916) **Description:** Contrib hosts two confmap providers which aren't in the local binary. We should add them so we can test them with a live system. cc @Aneurysm9 @driverpt @atoulme --------- Co-authored-by: Evan Bradley <[email protected]> * chore: remote write exporter retry on 429 (#31924) **Description:** <Describe what has changed.> This PR adds an option to retry the remote write requests when the receiving backend responds with 429 http status code, <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> **Link to tracking Issue:** #31032 **Testing:** <Describe what testing was performed and which tests were added.> Added tests covering the case. **Documentation:** <Describe the documentation added.> Not sure what's the pattern for documenting feature flags. --------- Co-authored-by: Anthony Mirabella <[email protected]> Co-authored-by: Pablo Baeyens <[email protected]> Co-authored-by: David Ashpole <[email protected]> * [chore] vcenterreceiver Adds Accidentally Removed Unit Test Configs/Results (#32987) **Description:** <Describe what has changed.> In this [PR](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/32913), I accidentally removed all the enabled configs from the 2nd set of unit tests. This was incorrect as there are still 4 of them that are currently disabled by default. This is rectifying that by adding those back in. **Link to tracking Issue:** <Issue number if applicable> **Testing:** <Describe what testing was performed and which tests were added.> This change is for the unit tests only **Documentation:** <Describe the documentation added.> NA * OpenTelemetry Protocol with Apache Arrow Exporter component (#31996) **Description:** This is the same code as OTel-Arrow at https://github.com/open-telemetry/otel-arrow/releases/tag/v0.23.0 (plus [backported lint and test fixes](https://github.com/open-telemetry/otel-arrow/commit/0910113d46454c80881db840e21f25485dce2499)). Only import statements change here, to match the host repository. **Link to tracking Issue:** #26491 **Testing:** Test coverage is approximately 90%. **Documentation:** I double-checked and the existing README had only a few updates needed. * [exporter/sumologic] change logs behavior (#32939) **Description:** * set OTLP as default format * add support for OTLP format * do not support metadata attributes * do not support source headers **Link to tracking Issue:** #32315 **Testing:** * unit tests **Documentation:** * inline comments * readme --------- Signed-off-by: Dominik Rosiek <[email protected]> * Fix diff to upstream * [chore][exporter/rabbitmq] Add missing code owner (#32984) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> @atoulme [volunteered to be the sponsor](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/28891#issuecomment-1981402878) of this component, so I believe he should be listed as a code owner. From [CONTRIBUTING.md](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#adding-new-components): ``` A sponsor is an approver who will be in charge of being the official reviewer of the code and become a code owner for the component. ``` **Link to tracking Issue:** <Issue number if applicable> #28891 * [chore][receiver/sqlserver] Document Windows-only metrics (#32944) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> The referenced metrics are being gathered from Windows perf counters, thus they're currently only available on Windows. With the introduction of direct connection and other supported OSs, I think it would be good to be clear to users about metric availability. * [receiver/splunkehecreceiver] Align acking behavior with that of Splu… (#32996) **Description:** - Make the channelID header case-insensitive - Make hecreceiver endpoints able to extract channelID from query params **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32995 * [receiver/vcenter] Collection Time Performance Enhancement (#32991) **Description:** <Describe what has changed.> There were already some improvements made as far as how networks calls were made centered around Virtual Machines. This allowed collection times to decrease from ~90s to ~27s in an environment with 1 Cluster, 2 Hosts, & 280 VMs. Making similar changes for all resource types helped to further decrease collection times. Now collection time has decreased from ~27s to <~3s for the same environment. Here's a general list of the changes made: - Now makes all network calls (per datacenter) first and stores returned data. - Processes this data afterwards to convert to OTEL resources/metrics (refactored to new file). - Moves all metric recording to metrics.go to keep consistent. - Moves all resource builder creation to resources.go to keep consistent. - Updates/fixes tests. **Link to tracking Issue:** <Issue number if applicable> #31837 Although this issue prescribes a solution to the problem (goroutines) which ended up not being necessary **Testing:** <Describe what testing was performed and which tests were added.> Unit Tests & Integration Tests Passing as well as Manual Testing in Local Environments **Documentation:** <Describe the documentation added.> N/A * [exporter/sumologic]: add sticky session (#33011) **Description:** Adds support for sticky session in order to better support AWS LB. This code is moved from Sumo Logic repository **Link to tracking Issue:** #32315 **Testing:** Tested manually **Documentation:** N/A --------- Signed-off-by: Dominik Rosiek <[email protected]> * [receiver/sqlserver] Add more metrics (#32932) **Description:** This change adds a query, scraper, and some more metrics for data from the performance counter SQL server table. The query itself is mostly taken from Telegraf's SQL server plugin. This also reuses the existing metric `sqlserver.lock.wait.rate`, so now it will be available both from Windows performance counters, as well as other OSs when directly connecting to the SQL server instance. **Naming and format feedback on new metrics would be greatly appreciated.** **Link to tracking Issue:** #29865 **Testing:** Added tests for the query itself, as well as the new scraper. Co-authored-by: Daniel Jaglowski <[email protected]> * deltatocumulative: exponential histograms (#32030) **Description:** Implements accumulation of exponential histograms by adding bucket-per-bucket. - [x] Align bucket offset to the smaller one - [x] Merge buckets by adding up each buckets count - [x] Widen zero buckets so they are the same - [x] Adjust scale to the lowest one **Link to tracking Issue:** https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/30705 **Testing:** Extensive tests have been added to the `internal/data` package **Documentation:** not needed * [pkg/stanza] Add container operator parser (#32594) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This PR implements the new container logs parser as it was proposed at https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/31959. **Link to tracking Issue:** <Issue number if applicable> https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/31959 **Testing:** <Describe what testing was performed and which tests were added.> Added unit tests. Providing manual testing steps as well: ### How to test this manually 1. Using the following config file: ```yaml receivers: filelog: start_at: end include_file_name: false include_file_path: true include: - /var/log/pods/*/*/*.log operators: - id: container-parser type: container output: m1 - type: move id: m1 from: attributes.k8s.pod.name to: attributes.val - id: some type: add field: attributes.key2.key_in value: val2 exporters: debug: verbosity: detailed service: pipelines: logs: receivers: [filelog] exporters: [debug] processors: [] ``` 2. Start the collector: `./bin/otelcontribcol_linux_amd64 --config ~/otelcol/container_parser/config.yaml` 3. Use the following bash script to create some logs: ```bash #! /bin/bash echo '2024-04-13T07:59:37.505201169-05:00 stdout P This is a very very long crio line th' >> /var/log/pods/kube-scheduler-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d3/kube-scheduler43/1.log echo '{"log":"INFO: log line here","stream":"stdout","time":"2029-03-30T08:31:20.545192187Z"}' >> /var/log/pods/kube-controller-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d6/kube-controller/1.log echo '2024-04-13T07:59:37.505201169-05:00 stdout F at is awesome! crio is awesome!' >> /var/log/pods/kube-scheduler-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d3/kube-scheduler43/1.log echo '2021-06-22T10:27:25.813799277Z stdout P some containerd log th' >> /var/log/pods/kube-scheduler-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d3/kube-scheduler44/1.log echo '{"log":"INFO: another log line here","stream":"stdout","time":"2029-03-30T08:31:20.545192187Z"}' >> /var/log/pods/kube-controller-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d6/kube-controller/1.log echo '2021-06-22T10:27:25.813799277Z stdout F at is super awesome! Containerd is awesome' >> /var/log/pods/kube-scheduler-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d3/kube-scheduler44/1.log echo '2024-04-13T07:59:37.505201169-05:00 stdout F standalone crio line which is awesome!' >> /var/log/pods/kube-scheduler-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d3/kube-scheduler43/1.log echo '2021-06-22T10:27:25.813799277Z stdout F standalone containerd line that is super awesome!' >> /var/log/pods/kube-scheduler-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d3/kube-scheduler44/1.log ``` 4. Run the above as a bash script to verify any parallel processing. Verify that the output is correct. ### Test manually on k8s 1. `make docker-otelcontribcol && docker tag otelcontribcol otelcontribcol-dev:0.0.1 && kind load docker-image otelcontribcol-dev:0.0.1` 2. Install using the following helm values file: ```yaml mode: daemonset presets: logsCollection: enabled: true image: repository: otelcontribcol-dev tag: "0.0.1" pullPolicy: IfNotPresent command: name: otelcontribcol config: exporters: debug: verbosity: detailed receivers: filelog: start_at: end include_file_name: false include_file_path: true exclude: - /var/log/pods/default_daemonset-opentelemetry-collector*_*/opentelemetry-collector/*.log include: - /var/log/pods/*/*/*.log operators: - id: container-parser type: container output: some - id: some type: add field: attributes.key2.key_in value: val2 service: pipelines: logs: receivers: [filelog] processors: [batch] exporters: [debug] ``` 3. Check collector's output to verify the logs are parsed properly: ```console 2024-05-10T07:52:02.307Z info LogsExporter {"kind": "exporter", "data_type": "logs", "name": "debug", "resource logs": 1, "log records": 2} 2024-05-10T07:52:02.307Z info ResourceLog #0 Resource SchemaURL: ScopeLogs #0 ScopeLogs SchemaURL: InstrumentationScope LogRecord #0 ObservedTimestamp: 2024-05-10 07:52:02.046236071 +0000 UTC Timestamp: 2024-05-10 07:52:01.92533954 +0000 UTC SeverityText: SeverityNumber: Unspecified(0) Body: Str(otel logs at 07:52:01) Attributes: -> log: Map({"iostream":"stdout"}) -> time: Str(2024-05-10T07:52:01.92533954Z) -> k8s: Map({"container":{"name":"busybox","restart_count":"0"},"namespace":{"name":"default"},"pod":{"name":"daemonset-logs-6f6mn","uid":"1069e46b-03b2-4532-a71f-aaec06c0197b"}}) -> logtag: Str(F) -> key2: Map({"key_in":"val2"}) -> log.file.path: Str(/var/log/pods/default_daemonset-logs-6f6mn_1069e46b-03b2-4532-a71f-aaec06c0197b/busybox/0.log) Trace ID: Span ID: Flags: 0 LogRecord #1 ObservedTimestamp: 2024-05-10 07:52:02.046411602 +0000 UTC Timestamp: 2024-05-10 07:52:02.027386192 +0000 UTC SeverityText: SeverityNumber: Unspecified(0) Body: Str(otel logs at 07:52:02) Attributes: -> log.file.path: Str(/var/log/pods/default_daemonset-logs-6f6mn_1069e46b-03b2-4532-a71f-aaec06c0197b/busybox/0.log) -> time: Str(2024-05-10T07:52:02.027386192Z) -> log: Map({"iostream":"stdout"}) -> logtag: Str(F) -> k8s: Map({"container":{"name":"busybox","restart_count":"0"},"namespace":{"name":"default"},"pod":{"name":"daemonset-logs-6f6mn","uid":"1069e46b-03b2-4532-a71f-aaec06c0197b"}}) -> key2: Map({"key_in":"val2"}) Trace ID: Span ID: Flags: 0 ... ``` **Documentation:** <Describe the documentation added.> Added Signed-off-by: ChrsMark <[email protected]> * fix(deps): update module google.golang.org/genproto/googleapis/api to v0.0.0-20240513163218-0867130af1f8 (#33039) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google.golang.org/genproto/googleapis/api](https://togithub.com/googleapis/go-genproto) | `v0.0.0-20240506185236-b8a5c65736ae` -> `v0.0.0-20240513163218-0867130af1f8` | [![age](https://developer.mend.io/api/mc/badges/age/go/google.golang.org%2fgenproto%2fgoogleapis%2fapi/v0.0.0-20240513163218-0867130af1f8?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/google.golang.org%2fgenproto%2fgoogleapis%2fapi/v0.0.0-20240513163218-0867130af1f8?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/google.golang.org%2fgenproto%2fgoogleapis%2fapi/v0.0.0-20240506185236-b8a5c65736ae/v0.0.0-20240513163218-0867130af1f8?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/google.golang.org%2fgenproto%2fgoogleapis%2fapi/v0.0.0-20240506185236-b8a5c65736ae/v0.0.0-20240513163218-0867130af1f8?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNTEuMiIsInVwZGF0ZWRJblZlciI6IjM3LjM1MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZWJvdCJdfQ==--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * [chore][receiver/sqlserver] Update documentation for lock wait rate metric (#33023) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> As a result of https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/32932 the metric `sqlserver.lock.wait.rate` is now available on Windows, or when directly connecting to a SQL server instance. Since this metric is always available, it no longer needs extended documentation about availability. * [exporter/datadog] Actually gzip host metadata (#32992) **Description:** We were sending a payload and setting the `Content-Encoding: gzip` HTTP header while actually sending uncompressed data. This has been happening for at least a couple years. This fixes the bug by actually gzipping the payload * [chore][pkg/stanza] Add syslog octet counting test cases with whitespace scenarios (#32832) Resolves #31477 --------- Co-authored-by: Tiffany Hrabusa <[email protected]> Co-authored-by: Curtis Robert <[email protected]> * fix(deps): update module github.com/grafana/loki/pkg/push to v0.0.0-20240514073905-7cc9a9386a8f (#33031) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/grafana/loki/pkg/push](https://togithub.com/grafana/loki) | `v0.0.0-20240507085123-772616cd8f5c` -> `v0.0.0-20240514073905-7cc9a9386a8f` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240514073905-7cc9a9386a8f?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240514073905-7cc9a9386a8f?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240507085123-772616cd8f5c/v0.0.0-20240514073905-7cc9a9386a8f?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fgrafana%2floki%2fpkg%2fpush/v0.0.0-20240507085123-772616cd8f5c/v0.0.0-20240514073905-7cc9a9386a8f?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Configuration 📅 **Schedule**: Branch creation - "on tuesday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/open-telemetry/opentelemetry-collector-contrib). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNTEuMiIsInVwZGF0ZWRJblZlciI6IjM3LjM1MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiLCJyZW5vdmF0ZWJvdCJdfQ==--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: opentelemetrybot <[email protected]> * fix(deps): update module github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common to v1.0.919 (#33036) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common](https://togithub.com/tencentcloud/tencentcloud-sdk-go) | `v1.0.914` -> `v1.0.919` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2ftencentcloud%2ftencentcloud-sdk-go%2ftencentcloud%2fcommon/v1.0.919?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/go/github.com%2ftencentcloud%2ftencentcloud-sdk-go%2ftencentcloud%2fcommon/v1.0.919?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/go/github.com%2ftencentcloud%2ftencentcloud-sdk-go%2ftencentcloud%2fcommon/v1.0.914/v1.0.919?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2ftencentcloud%2ftencentcloud-sdk-go%2ftencentcloud%2fcommon/v1.0.914/v1.0.919?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>tencentcloud/tencentcloud-sdk-go (github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common)</summary> ### [`v1.0.919`](https://togithub.com/tencentcloud/tencentcloud-sdk-go/blob/HEAD/CHANGELOG.md#Release-v10919) [Compare Source](https://togithub.com/tencentcloud/tencentcloud-sdk-go/compare/v1.0.918...v1.0.919) #### 负载均衡(clb) 版本:2018-03-17 ##### 第 105 次发布 发布时间:2024-05-14 01:28:52 本次发布包含了以下内容: 改善已有的文档。 修改数据结构: - [TargetGroupAssociation](https://cloud.tencent.com/document/api/214/30694#TargetGroupAssociation) - 新增成员:Weight #### TDSQL-C MySQL 版(cynosdb) 版本:2019-01-07 ##### 第 89 次发布 发布时间:2024-05-14 01:39:01 本次发布包含了以下内容: 改善已有的文档。 修改接口: - [CreateCLSDelivery](https://cloud.tencent.com/document/api/1003/106079) - 新增入参:InstanceId, CLSInfoList - 新增出参:TaskId - [DeleteCLSDelivery](https://cloud.tencent.com/document/api/1003/106078) - 新增入参:InstanceId, CLSTopicIds - 新增出参:TaskId - [DescribeInstanceCLSLogDelivery](https://cloud.tencent.com/document/api/1003/106077) - 新增入参:InstanceId - 新增出参:TotalCount, InstanceCLSDeliveryInfos - [StartCLSDelivery](https://cloud.tencent.com/document/api/1003/106076) - 新增入参:InstanceId, CLSTopicIds - 新增出参:TaskId - [StopCLSDelivery](https://cloud.tencent.com/document/api/1003/106075) - 新增入参:InstanceId, CLSTopicIds - 新增出参:TaskId 新增数据结构: - [CLSInfo](https://cloud.tencent.com/document/api/1003/48097#CLSInfo) - [InstanceCLSDeliveryInfo](https://cloud.tencent.com/document/api/1003/48097#InstanceCLSDeliveryInfo) #### 数据安全治理中心(dsgc) 版本:2019-07-23 ##### 第 11 次发布 发布时间:2024-05-14 01:46:03 本次发布包含了以下内容: 改善已有的文档。 修改数据结构: - [AKSKLeak](https://cloud.tencent.com/document/api/1087/96844#AKSKLeak) - <font color="#dd0000">**修改成员**:</font>AK, SK, URL - [AssetCosDetail](https://cloud.tencent.com/document/api/1087/96844#AssetCosDetail) - <font color="#dd0000">**修改成员**:</font>Bucket, DataType, FileNums, SensitiveFileNums, DistributionData, MatchedNum - [DSPACosMetaDataInfo](https://cloud.tencent.com/document/api/1087/96844#DSPACosMetaDataInfo) - <font color="#dd0000">**修改成员**:</font>BindStatus, Storage - [DbRelationStatusItem](https://cloud.tencent.com/document/api/1087/96844#DbRelationStatusItem) - <font color="#dd0000">**修改成员**:</font>DbName, BindStatus, ValidStatus - [DbTaskResult](https://cloud.tencent.com/document/api/1087/96844#DbTaskResult) - <font color="#dd0000">**修改成员**:</font>Result, ResultDescription, ErrDescription, ResourceId, DbName - [ErrDescription](https://cloud.tencent.com/document/api/1087/96844#ErrDescription) - <font color="#dd0000">**修改成员**:</font>ErrCode, ErrMessage - [SuggestRiskLevelMatrixItem](https://cloud.tencent.com/document/api/1087/96844#SuggestRiskLevelMatrixItem) - <font color="#dd0000">**修改成员**:</font>SensitiveLevel, VulnerabilityLevel, RiskName, RiskScore #### 弹性 MapReduce(emr) 版本:2019-01-03 ##### 第 63 次发布 发布时间:2024-05-14 01:50:50 本次发布包含了以下内容: 改善已有的文档。 修改数据结构: - [ImpalaQuery](https://cloud.tencent.com/document/api/589/33981#ImpalaQuery) - 新增成员:BackendsCount, FragmentInstancesCount, RemainingFragmentCount #### Elasticsearch Service(es) 版本:2018-04-16 ##### 第 58 次发布 发布时间:2024-05-14 01:51:30 本次发布包含了以下内容: 改善已有的文档。 修改数据结构: - [ServerlessSpace](https://cloud.tencent.com/document/api/845/30634#ServerlessSpace) - 新增成员:KibanaLanguage #### 文字识别(ocr) 版本:2018-11-19 ##### 第 135 次发布 发布时间:2024-05-14 02:17:52 本次发布包含了以下内容: 改善已有的文档。 修改接口: - [HKIDCardOCR](https://cloud.tencent.com/document/api/866/46919) - <font color="#dd0000">**修改入参**:</font>DetectFake - 新增出参:WarnCardInfos - [MLIDCardOCR](https://cloud.tencent.com/document/api/866/37656) - 新增出参:WarnCardInfos - [RecognizeIndonesiaIDCardOCR](https://cloud.tencent.com/document/api/866/75195) - 新增出参:WarnCardInfos - [RecognizeThaiIDCardOCR](https://cloud.tencent.com/document/api/866/48475) - 新增出参:WarnCardInfos #### TI-ONE 训练平台(tione) 版本:2021-11-11 ##### 第 59 次发布 发布时间:2024-05-14 02:40:26 本次发布包含了以下内容: 改善已有的文档。 新增数据结构: - [CBSConfig](https://cloud.tencent.com/document/api/851/75051#CBSConfig) 修改数据结构: - [DataConfig](https://cloud.tencent.com/document/api/851/75051#DataConfig) - 新增成员:CBSSource #### TI-ONE 训练平台(tione) 版本:2019-10-22 ##### 第 13 次发布 发布时间:2024-05-14 02:40:07 本次发布包含了以下内容: 改善已有的文档。 <font color="#dd0000">**删除接口**:</font> - UpdateNotebookLifecycleScript #### 实时音视频(trtc) 版本:2019-07-22 ##### 第 73 次发布 发布时间:2024-05-14 02:45:49 本次发布包含了以下内容: 改善已有的文档。 修改数据结构: - [McuWaterMarkText](https://cloud.tencent.com/document/api/647/44055#McuWaterMarkText) - 新增成员:Font #### 微服务引擎(tse) 版本:2020-12-07 ##### 第 66 次发布 发布时间:2024-05-14 02:46:25 本次发布包含了以下内容: 改善已有的文档。 修改数据结构: - [CloudNativeAPIGatewayNode](https://cloud.tencent.com/document/api/1364/54942#CloudNativeAPIGatewayNode) - 新增成员:Weight, IsDefaultWeight - <font color="#dd0000">**修改成员**:</font>ZoneId, Zone, GroupId, GroupName, Status - [NativeGatewayServerGroup](https://cloud.tencent.com/document/api/1364/54942#NativeGatewayServerGroup) - 新增成员:DefaultWeight ### [`v1.0.918`](https://togithub.com/tencentcloud/tencentcloud-sdk-go/blob/HEAD/CHANGELOG.md#Release-v10918) [Compare Source](https://togithub.com/tencentcloud/tencentcloud-sdk-go/compare/v1.0.917...v1.0.918) #### TDSQL-C MySQL 版(cynosdb) 版本:2019-01-07 ##### 第 88 次发布 发布时间:2024-05-13 01:14:47 本次发布包含了以下内容: 改善已有的文档。 新增接口: - [CreateCLSDelivery](https://cloud.tencent.com/document/api/1003/106079) - [DeleteCLSDelivery](https://cloud.tencent.com/document/api/1003/106078) - [DescribeInstanceCLSLogDelivery](https://cloud.tencent.com/document/api/1003/106077) - [StartCLSDelivery](https://cloud.tencent.com/document/api/1003/106076) - [StopCLSDelivery](https://cloud.tencent.com/document/api/1003/106075) #### 腾讯电子签企业版(ess) 版本:2020-11-11 ##### 第 169 次发布 发布时间:2024-05-13 01:18:00 本次发布包含了以下内容: 改善已有的文档。 新增接口: - [DescribeUserVerifyStatus](https://cloud.tencent.com/document/api/1323/106080) #### 轻量应用服务器(lighthouse) 版本:2020-03-24 ##### 第 60 次发布 发布时间:2024-05-13 01:21:46 本次发布包含了以下内容: 改善已有的文档。 修改数据结构: - [Instance](https://cloud.tencent.com/document/api/1207/47576#Instance) - 新增成员:LatestOperationStartedTime - [InstancePriceDetail](https://cloud…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
There is a lot of changes. I will leave this PR as a draft and will try to create some smaller PRs, which would help to review the code
Link to tracking Issue:
#31479
Testing:
Unit tests. This code is also used by our customers and we test it e2e internally
Documentation:
README.md