Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DNM] Test Otel Logging #1854

Draft
wants to merge 123 commits into
base: master
Choose a base branch
from
Draft
Changes from all commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
e0cc933
Implement an OTel version of the files receiver mixin
quentinmit Oct 17, 2023
0633ecd
Generate logging pipelines in OTel config file
quentinmit Oct 17, 2023
6efad97
Store experiments in Context for testing
quentinmit Oct 18, 2023
b0af7ea
Load experimental features from unit tests
quentinmit Oct 18, 2023
56cfef5
Correctly generate a file receiver
quentinmit Oct 18, 2023
1db00f0
Disable fluent-bit receivers when using otel logging
quentinmit Oct 20, 2023
cb7d4d0
Implement OTel support for parse_json
quentinmit Oct 20, 2023
f158387
Fix golden directory creation for new tests
quentinmit Oct 20, 2023
270a079
Add a parse_json otel test
quentinmit Oct 20, 2023
c455e0b
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Nov 20, 2023
c6c5bdd
Run fluent-bit transformation tests in a sub-test
quentinmit Nov 20, 2023
244ce32
Checkpoint on otel test that builds
quentinmit Nov 21, 2023
1233cc1
Attempt to run otel tests
quentinmit Nov 21, 2023
204619d
Generate most of an otel config
quentinmit Nov 21, 2023
fac9cff
Signal the collector to stop after 5 seconds
quentinmit Nov 22, 2023
7447414
Overwrite otel log timestamp
quentinmit Nov 22, 2023
99af4de
Golden otel output for transformation test
quentinmit Nov 22, 2023
e7911c9
Use .yaml for otel output file
quentinmit Nov 22, 2023
4effba5
Support OTLP logs via OTel
quentinmit Dec 11, 2023
efd7643
Set log name and resource_name for OTLP logs
quentinmit Dec 12, 2023
4e536a2
Update goldens
quentinmit Dec 12, 2023
e5fc9db
Add support for converting filter members to OTTL
quentinmit Dec 12, 2023
11393b9
Test OTTLAccessor
quentinmit Dec 12, 2023
3dfb2cb
Switch from logstransform to transform for parse_json
quentinmit Dec 12, 2023
e587114
Add validation for legacy `field` fields
quentinmit Dec 12, 2023
13b0ecc
One more field to validate
quentinmit Dec 13, 2023
a54234b
Move OTTL generation into a seperate package
quentinmit Dec 14, 2023
f34ca19
Remove old TODO
quentinmit Dec 14, 2023
bc70d6a
Move OTTL generation into its own package
quentinmit Dec 15, 2023
3eea853
Process collector errors and include in golden yaml
quentinmit Dec 15, 2023
6d2be12
Update transformation goldens
quentinmit Dec 15, 2023
25353a5
Implement most of modify_fields
quentinmit Dec 18, 2023
4da4cbc
Fix time format in transformation test
quentinmit Dec 18, 2023
9f5a2a7
Update to Go 1.21
quentinmit Dec 18, 2023
5cc4873
Update go-nvml to fix building with Go 1.21
quentinmit Dec 18, 2023
5143c42
Start implementing OTTL filter transformation
quentinmit Dec 18, 2023
5e7d516
Handle missing fields better
quentinmit Dec 19, 2023
13ba7c1
Convert kvlistValue to a map
quentinmit Dec 19, 2023
a2bbb92
Improve time handling
quentinmit Dec 19, 2023
50f5522
update goldens
quentinmit Dec 19, 2023
fac4392
Handle special fields in parse_json
quentinmit Dec 20, 2023
18636ea
Generate OTTL for filter expressions
quentinmit Dec 20, 2023
737ff08
Implement OTTL OmitIf
quentinmit Dec 20, 2023
007f265
Fix handling of labels
quentinmit Dec 20, 2023
c4a4067
Remove unused TimestampConfig
quentinmit Dec 20, 2023
df81b2b
Fix stacked modify_fields
quentinmit Dec 20, 2023
4167e1d
Support MapValues
quentinmit Dec 20, 2023
55eb440
Add confgenerator test for modify_fields
quentinmit Dec 20, 2023
23c991e
Run otelopscol transformation tests
quentinmit Dec 21, 2023
9952228
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Dec 21, 2023
832bada
Always generate config for transformation tests, even if binaries wer…
quentinmit Dec 21, 2023
3f6fd7a
Fix --update when golden files don't exist
quentinmit Dec 21, 2023
7b61c51
Handle otel config parsing failures
quentinmit Dec 21, 2023
e03ebdf
Add transformation test based on modify_fields integration test, fix …
quentinmit Dec 21, 2023
11eb6b7
Fix comment
quentinmit Jan 3, 2024
043e22c
Remove logging.googleapis.com/labels after merging it
quentinmit Jan 3, 2024
afdd90a
Import TestLogEntrySpecialFields from ops_agent_test
quentinmit Jan 3, 2024
3d5c79f
Use `modify_fields` to set the log name
quentinmit Jan 3, 2024
8e7bb56
Remove all kvlistValue, not just the labels
quentinmit Jan 4, 2024
99807fd
Provide a mock Cloud Logging endpoint for otel, so we can log the `Wr…
quentinmit Jan 5, 2024
99359a4
Fix timestamp handling for LogEntry
quentinmit Jan 6, 2024
79dac25
Regenerate goldens
quentinmit Jan 6, 2024
a83c7ea
Import TestProcessorOrder from ops_agent_test
quentinmit Jan 6, 2024
cb38064
Update otelopscol to support timezone and float parsing
quentinmit Jan 8, 2024
bbd97c1
Update otelopscol to merge master (no functional changes)
quentinmit Jan 8, 2024
2cf27e9
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Jan 8, 2024
1efd08c
Always start with an empty cache["value"]
quentinmit Jan 9, 2024
2446f1b
Regenerate config goldens
quentinmit Jan 9, 2024
9418cee
Use a hardcoded resource for transformation tests
quentinmit Jan 9, 2024
19a89fd
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Jan 11, 2024
2977fa9
Remove accidentally committed files
quentinmit Jan 11, 2024
835888e
Update new transformation test output
quentinmit Jan 11, 2024
1598e0c
Add missing newlines on EXPERIMENTAL_FEATURES
quentinmit Jan 11, 2024
5b2d7d0
Return an error from OTTLExpression
quentinmit Jan 11, 2024
b259693
Add Context parameters and error returns to OTelReceiver and OTelProc…
quentinmit Jan 11, 2024
e0a2ec2
Add config test with (currently) invalid Ruby regex
quentinmit Jan 11, 2024
2301c08
Use experimental_otel_logging for processor tests to clean up the gol…
quentinmit Jan 11, 2024
acecb51
Add test for fields only unsupported in otel
quentinmit Jan 11, 2024
07508aa
Fix transformation_test
quentinmit Jan 11, 2024
1e3252f
Update to committed otelopscol submodule
quentinmit Jan 18, 2024
da88e46
Make sure setting severity works with OTLP data that also sets severi…
quentinmit Jan 18, 2024
1a2c983
Support textPayload as a name for body
quentinmit Jan 18, 2024
4dbd2ec
Fix log file path in otel
quentinmit Jan 19, 2024
9004fea
Report process failures as golden errors, so they can be checked in
quentinmit Jan 19, 2024
7c67931
Report config errors as goldens, to allow transformation tests that o…
quentinmit Jan 19, 2024
aac50d8
Slightly friendlier error when a processor is not supported in OTel
quentinmit Jan 19, 2024
3ca70a1
Implement exclude_logs, fix span and trace ID handling, and update tr…
quentinmit Jan 19, 2024
3260e43
Update confgenerator goldens
quentinmit Jan 19, 2024
09865aa
Move severity_text special case into ottl
quentinmit Jan 19, 2024
c08e158
Allow trace to be set with a string that contains a projects/*/traces…
quentinmit Jan 19, 2024
3c68f21
Fix otel transformation tests so they pass without --update
quentinmit Jan 19, 2024
74be1ec
Don't reuse constants from code-under-test when the constant is part …
quentinmit Jan 19, 2024
bcc70fc
logger is already a main logger
quentinmit Jan 19, 2024
3e4214c
Update opentelemetry-operations-collector
quentinmit Jan 19, 2024
af8fcc1
Run otel transformation tests with a fized timezone
quentinmit Jan 19, 2024
c91174c
Merged version of otelopscol
quentinmit Jan 19, 2024
5518689
Wait for 1s after the last log was received, to speed up tests
quentinmit Jan 19, 2024
f555d8d
Kill the collector after the file receiver has polled twice, instead …
quentinmit Jan 19, 2024
c88f6d5
Fix log.file.path location
quentinmit Jan 19, 2024
dc2343b
Remove even single-digit addresses because they are non-deterministic
quentinmit Jan 19, 2024
9cf1548
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Jan 22, 2024
c7e74f4
Ignore errors in transformprocessor and filterprocessor
quentinmit Jan 22, 2024
869e9cf
Add test for unparsable times, to make sure error_mode works
quentinmit Jan 22, 2024
f038bc7
Match fluent-bit behavior when parsing timestamps
quentinmit Jan 22, 2024
6881476
Remove done TODOs
quentinmit Jan 22, 2024
7f57ebd
Implement bare-bones windows event log receiver
quentinmit Jan 23, 2024
69e9c02
Add windowseventlogreceiver to otelopscol
quentinmit Jan 24, 2024
51d1b83
Initial attempt to generate windowseventlog v1 from otel
quentinmit Jan 27, 2024
a9e02f8
Handle event source in windows event logs
quentinmit Jan 29, 2024
db7486e
Update collector
quentinmit Jan 29, 2024
de1b028
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Jan 29, 2024
14aa4d0
Add `otlp_logging` experiment flag to enable logging support in the o…
quentinmit Jan 30, 2024
0b52ed4
Add new EXPERIMENTAL_FEATURES file
quentinmit Jan 30, 2024
f21d072
Fix OTTL != and add transformation test to confirm
quentinmit Jan 31, 2024
a626a3f
Fix typo
quentinmit Jan 31, 2024
6712ecb
Refactor config generation to allow per-pipeline otel/fluentbit decis…
quentinmit Feb 2, 2024
c52ab50
Fix self_metrics to handle combined logging receivers
quentinmit Feb 2, 2024
4b6f965
Set Sid to an empty string by default, and use CustomConvertFunc for …
quentinmit Feb 5, 2024
53cb2c2
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Feb 5, 2024
02783e6
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Feb 5, 2024
d8d7d34
Use Go 1.21's maps package
quentinmit Feb 5, 2024
955c7b3
Merge remote-tracking branch 'origin/master' into otel-logging
quentinmit Feb 5, 2024
f6caac8
Test.
franciscovalentecastro Jan 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Ops Agents

Building & Testing PR.

This repository contains the Ops Agents (Linux and Windows) that are part of the
Google Cloud Operations product suite (specifically Cloud Logging and Cloud
Monitoring).
Expand Down
Loading