Transform processor cannot operate on body if it is a map

[sumo-drosiek]

Component(s)

processor/transform

What happened?

Description

I would like to move attribute from body (which is a map) to the resource attributes

Here is my reproduction log file:

{"timestamp":"2023-05-01T14:57:19.807Z","level":"info"}

Steps to Reproduce

Run attached configuration

Expected Result

resource level attribute level is set to info

Actual Result

resource level attribute level is set to full body

Collector version

v0.77.0

Environment information

Environment

Unrelated

OpenTelemetry Collector configuration

exporters:
  logging:
    verbosity: detailed
receivers:
  filelog:
    start_at: beginning
    include:
    - ./tmp/logs/2.json
    operators:
      - if: body matches "^{[\\s\\S]+"
        parse_from: body
        parse_to: body
        type: json_parser

processors:
  transform:
    log_statements:
      context: log
      statements:
        - set(resource.attributes["level"], body["level"])
service:
  pipelines:
    logs/containers:
      exporters:
      - logging
      processors:
      - transform
      receivers:
      - filelog

Log output

2023-05-18T11:07:02.751+0200	info	service/telemetry.go:113	Setting up own telemetry...
2023-05-18T11:07:02.751+0200	info	service/telemetry.go:136	Serving Prometheus metrics	{"address": ":8888", "level": "Basic"}
2023-05-18T11:07:02.751+0200	info	[email protected]/exporter.go:286	Development component. May change in the future.	{"kind": "exporter", "data_type": "logs", "name": "logging"}
2023-05-18T11:07:02.753+0200	info	service/service.go:141	Starting otelcol-contrib...	{"Version": "0.77.0", "NumCPU": 16}
2023-05-18T11:07:02.753+0200	info	extensions/extensions.go:41	Starting extensions...
2023-05-18T11:07:02.753+0200	info	adapter/receiver.go:56	Starting stanza receiver	{"kind": "receiver", "name": "filelog", "data_type": "logs"}
2023-05-18T11:07:02.753+0200	info	service/service.go:158	Everything is ready. Begin running and processing data.
2023-05-18T11:07:02.954+0200	info	fileconsumer/file.go:196	Started watching file	{"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "path": "tmp/logs/2.json"}
2023-05-18T11:07:03.054+0200	info	LogsExporter	{"kind": "exporter", "data_type": "logs", "name": "logging", "resource logs": 1, "log records": 1}
2023-05-18T11:07:03.054+0200	info	ResourceLog #0
Resource SchemaURL: 
Resource attributes:
     -> level: Map({"level":"info","timestamp":"2023-05-01T14:57:19.807Z"})
ScopeLogs #0
ScopeLogs SchemaURL: 
InstrumentationScope  
LogRecord #0
ObservedTimestamp: 2023-05-18 09:07:02.954294022 +0000 UTC
Timestamp: 1970-01-01 00:00:00 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Map({"level":"info","timestamp":"2023-05-01T14:57:19.807Z"})
Attributes:
     -> log.file.name: Str(2.json)
Trace ID: 
Span ID: 
Flags: 0
	{"kind": "exporter", "data_type": "logs", "name": "logging"}
2023-05-18T11:07:04.255+0200	info	otelcol/collector.go:238	Received signal from OS	{"signal": "interrupt"}
2023-05-18T11:07:04.255+0200	info	service/service.go:167	Starting shutdown...
2023-05-18T11:07:04.255+0200	info	adapter/receiver.go:150	Stopping stanza receiver	{"kind": "receiver", "name": "filelog", "data_type": "logs"}
2023-05-18T11:07:04.264+0200	info	extensions/extensions.go:55	Stopping extensions...
2023-05-18T11:07:04.264+0200	info	service/service.go:181	Shutdown complete.

Additional context

No response

[github-actions]

Pinging code owners:

• processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[BinaryFissionGames]

Related issue: #17396

[evan-bradley]

Thanks for reporting this. I see two issues here:

1. As you said, the body can't be indexed. We should allow this, since internally the body is just a pcommon.Value.
2. We don't check that non-indexable values aren't indexed, so paths like time_unix_nano["test"][0][1] don't throw an error right now.

I'll open PRs to address each of these.

[TylerHelmuth]

I do think we should stick to our indexing rules of slices and maps when using it on body. If body is a pcommon.ValueTypeStr or pcommon.ValueTypeInt then indexing it should error.

[evan-bradley]

We don't check that non-indexable values aren't indexed, so paths like time_unix_nano["test"][0][1] don't throw an error right now.

Fixing this isn't trivial, so I opened a new issue to track this: #22744.