-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[jsonlogencodingextension] Add Modes for Body with Attributes #32722
[jsonlogencodingextension] Add Modes for Body with Attributes #32722
Conversation
Signed-off-by: Blake R <[email protected]>
Signed-off-by: Blake R <[email protected]>
Signed-off-by: Blake R <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am ok with this as an alternative to otlp json encoding. Noting outputting the resource attributes for each record is of course wasteful, I suppose you meant for this to be a self sufficient log representation
If I understand your comment enough that is correct. The idea is each index in the array is a self contained log. The hope is that this provides a fairly stable format for ingestion to be able to parse into a usable format. The other alternative I guess that I wasn't sure if it was possible would be to have a seperate return for each log record therefore instead of an array of items you'd have {
"body": "log testing",
"resource": {
"test": "logs-test"
}
} which I think may be desirable however I am not sure the value add is worth it. I think if anything a future enhancement to the encoder could be a "split" mode where a user could define whether they'd want to split it into two separate log entries or have it combined into one but I feel like that may not be an encoder issue rather a processor problem. We could also have a seperate {
"body": [
"foo",
"bar",
],
"resource": {
"test": "logs-test"
}
} |
Signed-off-by: Blake R <[email protected]>
Attributes map[string]any `json:"attributes,omitempty"` | ||
Resource map[string]any `json:"resource,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe log_attributes
and resource_attributes
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking of trying to stay with camelCase as most json is but I wouldn't be opposed to logAttributes
and resourceAttributes
Signed-off-by: Blake R <[email protected]>
Signed-off-by: Blake R <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM with 2 nits.
Signed-off-by: Blake R <[email protected]>
BTW, your approach discards scope log attributes, I'm not sure if that matters. |
Signed-off-by: Blake R <[email protected]>
I don't think it matters but I am happy to hear other opinions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a few grammar and style suggestions from a docs perspective. Thanks!
Signed-off-by: Blake R <[email protected]>
I believe I addressed all the naming and doc suggestions thank you for that feedback! I am not sure why there was so many status failures on the last commit. A lot of them seemed relating to something else so not sure it is because of this PR. I was able to validate linting and tests for this component were all passing. |
Hey folks just wondering what the next steps are to get this merged? 😄 |
{ | ||
"body": "log testing", | ||
"resource": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be resourceAttributes
I'm guessing?
"resource": { | |
"resourceAttributes": { |
### Mode | ||
|
||
#### body Mode |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be nice to make the headings consistent
#### body Mode | |
#### body |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be great if you could fix the small docs remarks in a follow-up PR @blakeromano
The component is in development stability, so it's probably better to move fast in small increments rather than sweat over every little detail.
Description:
The goal of this PR is to add configuration to the JSON Encoding method to have metadata from resources and attributes in logs and output those into a json attribute. This also will output the body of the log in the
body
attribute. This follows a similar pattern that the AWS Cloudwatch Log Exporter uses but removes any of the AWS specific logic and makes this a more generalized solutionLink to tracking Issue: #32679
Testing: Unit Tests were added to validate that the output is desired and that the existing logic remains the same.
Documentation: Documentation regarding the usage of the configuration as well as the output to expect is added.