Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

processor_opentelemetry_envelope: new processor to package non OTel Logs content as OTel #9001

Merged
merged 4 commits into from
Jun 24, 2024

Conversation

edsiper
Copy link
Member

@edsiper edsiper commented Jun 24, 2024

When collecting data from all plugins except in_opentelemetry, the records comes with basic timestamp, metadata and content; there are cases where this information collected needs to send to be send an OpenTelemetry endpoint (vendor or another OTel compatible endpoint) and packaging with proper OTel Log Resources and Scopes simplify the data transformation.

This processor creates the internal group with OTel basic structure. Note that this creates the envelope, for further processing can be used in conjunction with content modifier processor.

usage example:

pipeline:
  inputs:
    - name: dummy
      samples: 1

      processors:
        logs:
          - name: opentelemetry_envelope

          - name: content_modifier
            context: otel_resource_attributes
            action: upsert
            key: "aaa"
            value: "bbb"

  outputs:
    - name : stdout
      match: '*'

    - name: opentelemetry
      match: '*'
      host: 127.0.0.1
      port: 4318
      logs_uri: /v1/logs

note: there are other small commits in this PR to enable this processor.


Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

edsiper added 4 commits June 23, 2024 21:54
…ource as OTel

When collecting data from all plugins except in_opentelemetry, the records comes with
basic timestamp, metadata and content; there are cases where this information collected
needs to send to an OpenTelemetry endpoint (vendor or another OTel compatible endpoint)
and packaging with proper OTel Log Resources and Scopes simplify the data transformation.

This processor creates the internal group with OTel basic structure. Note that this creates
the envelope, for further processing can be used in conjunction with content modifier processor.

usage example:

    pipeline:
      inputs:
        - name: dummy
          samples: 1

          processors:
            logs:
              - name: opentelemetry_envelope

              - name: content_modifier
                context: otel_resource_attributes
                action: upsert
                key: "aaa"
                value: "bbb"

      outputs:
        - name : stdout
          match: '*'

        - name: opentelemetry
          match: '*'
          host: 127.0.0.1
          port: 4318
          logs_uri: /v1/logs

Signed-off-by: Eduardo Silva <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-required ok-package-test Run PR packaging tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants