Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use custom data type and custom JSON serialization for traceid #1840

Conversation

tigrannajaryan
Copy link
Member

Contributes to #1177

  1. The TraceID type uses custom data type so that JSON serialization
    is in hex format instead of base64 (which is the default Protobuf JSON
    format). Hex format is required by OTLP spec:
    https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/protocol/otlp.md#request
    SpanID must be also modified similarly. Will be done in a future PR
    to avoid creating a huge PR.

  2. Moved pdata.TraceID to its own file. Note that there is pdata.TraceID which
    is different from otlp TraceID custom data type. Due to the way packages are
    structured we need both to keep OTLP generated data types decoupled from pdata
    data types.

The majority of the changes in this commit are simply type changes from
[]byte to TraceID.

Contributes to open-telemetry#1177

1. The TraceID type uses custom data type so that JSON serialization
is in hex format instead of base64 (which is the default Protobuf JSON
format). Hex format is required by OTLP spec:
https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/protocol/otlp.md#request
SpanID must be also modified similarly. Will be done in a future PR
to avoid creating a huge PR.

2. Moved pdata.TraceID to its own file. Note that there is pdata.TraceID which
is different from otlp TraceID custom data type. Due to the way packages are
structured we need both to keep OTLP generated data types decoupled from pdata
data types.

The majority of the changes in this commit are simply type changes from
[]byte to TraceID.
@tigrannajaryan tigrannajaryan marked this pull request as draft September 23, 2020 20:30
@tigrannajaryan tigrannajaryan changed the title [WIP] Use custom data type and custom JSON serialization for traceid Use custom data type and custom JSON serialization for traceid Sep 23, 2020
@tigrannajaryan tigrannajaryan marked this pull request as ready for review September 23, 2020 20:31
@codecov
Copy link

codecov bot commented Sep 23, 2020

Codecov Report

Merging #1840 into master will decrease coverage by 0.00%.
The diff coverage is 98.24%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1840      +/-   ##
==========================================
- Coverage   91.12%   91.11%   -0.01%     
==========================================
  Files         263      264       +1     
  Lines       16105    16116      +11     
==========================================
+ Hits        14675    14684       +9     
- Misses       1003     1004       +1     
- Partials      427      428       +1     
Impacted Files Coverage Δ
consumer/pdata/trace.go 100.00% <ø> (ø)
...or/tailsamplingprocessor/sampling/always_sample.go 50.00% <ø> (ø)
...ilsamplingprocessor/sampling/numeric_tag_filter.go 70.00% <ø> (ø)
...or/tailsamplingprocessor/sampling/rate_limiting.go 0.00% <ø> (ø)
...ailsamplingprocessor/sampling/string_tag_filter.go 85.71% <ø> (ø)
service/internal/resources.go 25.33% <ø> (ø)
translator/internaldata/oc_to_traces.go 85.46% <ø> (ø)
...mplingprocessor/tailsamplingprocessor/processor.go 66.66% <66.66%> (ø)
consumer/pdata/generated_log.go 100.00% <100.00%> (ø)
consumer/pdata/generated_trace.go 100.00% <100.00%> (ø)
... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c32f36...d1c3838. Read the comment docs.

@@ -105,17 +105,6 @@ func (td Traces) ResourceSpans() ResourceSpansSlice {
return newResourceSpansSlice(td.orig)
}

type TraceID []byte
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not good, the otlpcommon.TraceID is not exposed, so users cannot set new TraceID. You can have a type TraceID = otlpcommon.TraceID and use the TraceID in the pdata public API.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's exactly what I have. See consumer/pdata/traceid.go line 22.

See PR description:

Moved pdata.TraceID to its own file.

@tigrannajaryan tigrannajaryan merged commit 75b1ac1 into open-telemetry:master Sep 24, 2020
@tigrannajaryan tigrannajaryan deleted the feature/tigran/custom-traceid-bytes3 branch September 24, 2020 14:13
@keitwb
Copy link
Contributor

keitwb commented Sep 24, 2020

@tigrannajaryan is there an issue to track the fixes in contrib? This broke the contrib-test in this repo.

@tigrannajaryan
Copy link
Member Author

No issue, I am already fixing this. This was expected. Will open a PR in a few.

@tigrannajaryan
Copy link
Member Author

@tigrannajaryan
Copy link
Member Author

@keitwb fix merged.

MovieStoreGuy pushed a commit to atlassian-forks/opentelemetry-collector that referenced this pull request Nov 11, 2021
* Added Linux-specific detector for the os.description attribute

* Generalized OS description detector with placeholder function for unimplemented OSes

* Extended osDescription function to *nix OSes based on golang.org/x/sys/unix

* Added WithOS resource configuration function to configure all of the OS resource attributes

* Implemented osDescription funtion for Windows OS

* Improved documentation header for *nix version of the osDescription function

* Added support for reading os-release file

* Added/updated documentation headers for *nix implementation of osDescription and related functions

* Changelog update

* Added support for reading macOS version information

* Mock approach to test OS description attribute

* Extracted common function getFirstAvailableFile to read the first available file from a list of candidates

* Upgraded golang.org/x/sys

* Changelog update

* Fixed wrong function name in documentation header for WithOSDescription

* Updated documentation header for platformOSDescription function

* Renamed restoreProcessAttributesProviders test helper function

The function restoreProcessAttributesProviders was renamed to simply
restoreAttributesProviders to better reflect its broader scope, which
not only applies to process attribute's providers.

* Fixed os_linux.go overriding build tags defined inside the file

The suffix on os_linux.go was overriding the build tags already defined
in that file. The file was renamed to os_release_unix.go, reflecting
the main function defined in the file.

For consistency, os_darwin.go was renamed to os_release_darwing.go, as
its primary purpose is to also define the osRelease function.

* Removed use of discontinued function resource.WithoutBuiltin

* Added PR number to changelog entries

* Updated go.sum files after run of make lint

* Linux implementation: ignore lines with an empty key

* Linux implementation: avoid unquoting strings less than two chars

* WIP: added tests for Linux support functions

* WIP: added tests for charsToString and getFirstAvailableFile functions

* Replaced os.CreateTemp with ioutil.TempFile as the former only exists in Go 1.16

* Added unameProvider type to decouple direct reference to unix.Uname function inside Uname()

* Added tests for Uname() function

* Replaced *os.File with io.Reader in parseOSReleaseFile to ease testing

* Added tests for parseOSReleaseFile function

* Darwin implementation: added tests for buildOSRelease function

* Replaced *os.File with io.Reader in parsePlistFile to ease testing

* Darwin implementation: added tests for parsePlistFile function

* Type in documentation header for Linux osRelease function

* Extracted logic for reading specific registry values into helper functions

* Added basic tests for Windows version of platformOSDescription and helper functions

* Manually formatted uint64 to strings to have an uniform interface for test assertions

* Asserts there's no error when opening registry key for testing

Co-authored-by: Robert Pająk <[email protected]>

* Simplified subtests by using a single test with multiple asserts

* go.sum update after running make

* Fix typo

Co-authored-by: Tyler Yahn <[email protected]>

* WIP: added placeholder implementation of platformOSDescription for unsupported OSes

* Fixed typo on osRelease documentation header

Co-authored-by: Chris Bandy <[email protected]>

* Fixed typo on test case name for ParsePlistFile tests

Co-authored-by: Chris Bandy <[email protected]>

* Linter fix in changelog

* go.sum updates after running make

* Used strings.Replacer instead of multiple strings.ReplaceAll calls

* Optimized implementation of charsToString

* Safer temporary file deletion with t.TempDir()

* Used t.Cleanup() for safer mocking of runtime providers in OS resource tests

* Handled optionality of DisplayVersion registry key.

For example, CI machine runs on:
Windows Server 2019 Datacenter (1809) [Version 10.0.17763.1999]

So, to not add an extra white space due to missing DisplayVersion,
this value is checked to be not empty, and only in such case a
trailing space is added for that component.

* Workaround to handle the case of DisplayVersion registry key not present

* Excluded unsupported GOOSes by negation of supported ones

* go.sum update after running make

Co-authored-by: Anthony Mirabella <[email protected]>
Co-authored-by: Robert Pająk <[email protected]>
Co-authored-by: Tyler Yahn <[email protected]>
Co-authored-by: Chris Bandy <[email protected]>
hughesjj pushed a commit to hughesjj/opentelemetry-collector that referenced this pull request Apr 27, 2023
Bumps [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action) from 0.6.1 to 0.6.2.
- [Release notes](https://github.com/aquasecurity/trivy-action/releases)
- [Commits](aquasecurity/trivy-action@0.6.1...0.6.2)

---
updated-dependencies:
- dependency-name: aquasecurity/trivy-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants