Adding log attributes to resource attributes

[sairamsadanala]

Component(s)

processor/resource

What happened?

Description

Setting up otelcol-contrib abstraction layer to receive telemetry having multiple exporter (loki and Splunk). Otel-hub extracts Kubernetes namespace from log body using Transform processor and set loki.tenant hint so that loki exporter can send to Loki endpoint. Logs need to be routed b/w loki and Splunk based on namespace attributes. Currently i have the namespace in log attributes but Routing processor only support context(header) or resource attributes and hence trying to set resource attribute from log attribute.

Steps to Reproduce

transform:
error_mode: ignore
log_statements:
- context: log
statements:
- set(attributes["cache"], ParseJSON(body)) where IsMatch(body[""], "^\{")
- set(attributes["tenant"], attributes["cache"]["kubernetes"]["namespace_name"])
- set(attributes["loki.tenant"], "tenant")
- delete_key(attributes, "cache")
resource:
attributes:
- key: tenantname
value: tenant
action: insert

Expected Result

tenantname resource attribute should be available.

Actual Result

No resource attributes are available..

Collector version

0.82

Environment information

Environment

OS: AWS EKS

OpenTelemetry Collector configuration

exporters:
  logging:
    verbosity: detailed
  loki:
    endpoint: http://XXXX/loki/api/v1/push
    retry_on_failure:
      enabled: true
      initial_interval: 1s
      max_elapsed_time: 120s
      max_interval: 300s
    sending_queue:
      storage: file_storage/psq
    tls:
      insecure: false
      insecure_skip_verify: true
  splunk_hec/logs:
    endpoint: https://XXXXXXXXXX/services/collector
    retry_on_failure:
      enabled: true
      initial_interval: 10s
      max_elapsed_time: 60s
      max_interval: 60s
    sending_queue:
      storage: file_storage/psq
    timeout: 30s
    tls:
      insecure: false
      insecure_skip_verify: true
    token: 95c4beb5-c1b0-4735-bcd3-310cea0c9fa8
extensions:
  health_check:
    endpoint: 0.0.0.0:13133
processors:
  memory_limiter:
    check_interval: 5s
    limit_percentage: 80
    spike_limit_percentage: 25
 
  transform:
    error_mode: ignore
    log_statements:
    - context: log
      statements:
      - set(attributes["cache"], ParseJSON(body)) where IsMatch(body[""], "^\\{")
      - set(attributes["tenant"], attributes["cache"]["kubernetes"]["namespace_name"])
      - set(attributes["loki.tenant"], "tenant")
      - delete_key(attributes, "cache")
    resource:
      attributes:
      - key: tenantname
        value: tenant 
        action: insert 
receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
        include_metadata: true
service:
  extensions:
  - health_check
  pipelines:
    logs:
      exporters:
      - logging
      - loki
      processors:
      - transform
      - resource
      receivers:
      - otlp

Log output

Not seeing resource attribute(tenantname) which was being set in resource processor.

Additional context

No response

[github-actions]

Pinging code owners:

• processor/resource: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[crobert-1]

Hello @sairamsadanala, have you been able to find a resolution to this issue yet?

I believe from the contexts documentation in the transform processor that you should be able to use the logs context to insert a resource attribute directly. From the logs context documentation, it looks like you should have direct resource attribute access. This should work instead of using two steps to insert an attribute to the log, then moving it from the log's attributes to the log's resource attributes.

Can you try to set the resource attribute directly from within the log context?

[github-actions]

Pinging code owners for processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[sairamsadanala]

Thanks Curtis Robert for your help.

…

On Fri, Oct 6, 2023 at 2:11 PM Curtis Robert ***@***.***> wrote: Hello @sairamsadanala <https://github.com/sairamsadanala>, have you been able to find a resolution to this issue yet? I believe from the contexts <https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor#contexts> documentation in the transform processor that you should be able to use the logs context to insert a resource attribute directly. From the logs context <https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/contexts/ottllog#paths> documentation, it looks like you should have direct resource attribute access. This should work instead of using two steps to insert an attribute to the log, then moving it from the log's attributes to the log's resource attributes. Can you try to set the resource attribute directly from within the log context? — Reply to this email directly, view it on GitHub <#26989 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKRJ4I5LAJZSMKVDOWE2YVDX6BJWXAVCNFSM6AAAAAA45KUNVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRGI4TAMZYG4> . You are receiving this because you were mentioned.Message ID: <open-telemetry/opentelemetry-collector-contrib/issues/26989/1751290387@ github.com>

[TylerHelmuth]

It sounds like this issue is resolved. Please ping me if you're still experiencing issues

[omri-shilton]

@sairamsadanala did you end up solving this issue? if so can you share the otel config you used?