feat(sources): opentelemetry log #13320
Conversation
|
|
github-actions
bot
added
the
domain: sources
✅ Deploy Preview for vector-project canceled.
|
|
@StephenWakely I've tidied up the code, and please CR again |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Thanks a lot for pursuing the effort.
Two major things, the source is names opentelemetry which implies that this source would be extended to support otel metrics (and traces) at some point, so IMHO the source config struct name should not refer to logs (or metrics/traces).
Can you also bootstrap documentation for this new source (inside ./website/cue/reference/components/sources/opentelemetry.cue) ?
Can I close #13135 as this PR seems to have precedence now ?
Thanks !
…opentelemetry-add-log
…nto opentelemetry-add-log
github-actions
bot
added
the
domain: external docs
This comment was marked as outdated.
This comment was marked as outdated.
|
Hi @prognant , I've updated the code, please review again, thx |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
…nto opentelemetry-add-log
github-actions
bot
added
domain: core
|
I don't know why there're two CI failures. I passed the examination in my dev environment by |
This comment was marked as outdated.
This comment was marked as outdated.
Soak Test ResultsBaseline: 9a9340a ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%: Fine details of change detection per experiment.
|
…opentelemetry-add-log
Any code to learn from? I didn't find any exmaple to test named output... |
You may find a test that asserts multiple outputs behaviour here |
|
I should have a PR with final updates and working tests in the next hour or so. 👍 |
I've commited a pr which convert the timestamp and have the unit test passed |
Signed-off-by: Spencer Gilbert <[email protected]>
Signed-off-by: Spencer Gilbert <[email protected]>
Signed-off-by: Spencer Gilbert <[email protected]>
Signed-off-by: Spencer Gilbert <[email protected]>
Soak Test ResultsBaseline: fd0b6c1 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%: Fine details of change detection per experiment.
|
Signed-off-by: Spencer Gilbert <[email protected]>
Soak Test ResultsBaseline: e8f32c3 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%: Fine details of change detection per experiment.
|
Signed-off-by: Spencer Gilbert <[email protected]>
Soak Test ResultsBaseline: e8f32c3 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%: Fine details of change detection per experiment.
|
Soak Test ResultsBaseline: e8f32c3 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%: Fine details of change detection per experiment.
|
Soak Test ResultsBaseline: e8f32c3 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%: Fine details of change detection per experiment.
|
|
Glad to see this merged,we can continue to support log sink and otel trace |
|
Could someone elaborate which version of Vector will contain this enhancement? I want to experiment with it ASAP as a means to workaround the lack of a Datadog log exporter in OTel collector. |
|
You can find it under the Nightly Builds: https://github.com/vectordotdev/vector/actions/workflows/nightly.yml |
Or You can try docker image(latest, nightly) directly from: https://hub.docker.com/r/timberio/vector/tags, |
|
@julealgon - I'd also expect this to be released on a stable release with If you're using a nightly version, the docs are available here. |
Example:
the opentelemetry log will be converted into EventLog:
I've been testing this source in our production for a week, and it works well! Hope this feature can be merged and I can continue to work on opentelemetry-metrics