[logstransform] logstransform is not in the valid processors

[Dongqi-Guo]

Component(s)

No response

What happened?

Description

I tried to use logstransform processor to handle log body from otel agent and then transfer to loki. I configured and started it. the console log shows it is not a valid processor

Steps to Reproduce

Expected Result

Actual Result

Collector version

v0.63

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")
centos

OpenTelemetry Collector configuration

processors:
  logstransform:
    operators:
      - type: regex_parser
        regex: 'INFO  \[(?P<traceId>\w*),(?P<agreementNo>\w*)\] (?P<logger>.*) - (?P<httpMethod>.*),(?P<uri>.*),(?P<uri2>.*),(?P<uriDesc>.*),(?P<state>.*),(?P<stateCode>.*),(?P<stateDesc>.*),(?P<responseTime>.*)'
        traceId:
          parse_from: body.traceId

Log output

* error decoding 'processors': unknown processors type: "logstransform" for id: "logstransform" (valid values: [redaction spanmetrics memory_limiter groupbyattrs k8sattributes metricstransform resource tail_sampling attributes filter experimental_metricsgeneration probabilistic_sampler span transform cumulativetodelta groupbytrace resourcedetection routing batch deltatorate servicegraph])

Additional context

No response

[github-actions]

Pinging code owners: @djaglowski @dehaansa. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[fatsheep9146]

seems that the logstranform is not default complied into contrib
https://github.com/open-telemetry/opentelemetry-collector-releases/blob/main/distributions/otelcol-contrib/manifest.yaml#L77-L100

is there any other processors which is able to support the function like regex_parser ?

@djaglowski

[djaglowski]

logstransformprocessor is considered experimental, so it has not been included in the contrib build. It's capabilities will eventually be reimplemented in transformprocessor.

It may be possible to include it in contrib temporarily, but it will eventually be deprecated. I cannot speak for other maintainers, but I would support this if we first resolve known issues such as #15378 and #9761.

[Dongqi-Guo]

@djaglowski
I encountered another problem.
I built a custom package included logstransform. when I started, also met a new issue as below:

2022-11-04T18:16:36.114+0800 info [email protected]/processor.go:56 Stopping logs transform processor {"kind": "processor", "name": "logstransform", "pipeline": "logs"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x101c28986]

goroutine 1 [running]:
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/pipeline.(*DirectedPipeline).Stop(0x0)
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/[email protected]/pipeline/directed.go:58 +0x46
github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor.(*logsTransformProcessor).Shutdown(0xc000b2c180, {0xc000b2a660?, 0xc000b0edc0?})
github.com/open-telemetry/opentelemetry-collector-contrib/processor/[email protected]/processor.go:57 +0x49
go.opentelemetry.io/collector/component.ShutdownFunc.Shutdown(...)
go.opentelemetry.io/[email protected]/component/component.go:93
go.opentelemetry.io/collector/service/internal/pipelines.(*Pipelines).ShutdownAll(0xc000b24aa0, {0x103d90988, 0xc0000580c0})
go.opentelemetry.io/[email protected]/service/internal/pipelines/pipelines.go:128 +0x43a
go.opentelemetry.io/collector/service.(*service).Shutdown(0xc000c35e00, {0x103d90988, 0xc0000580c0})
go.opentelemetry.io/[email protected]/service/service.go:121 +0xd4
go.opentelemetry.io/collector/service.(*Collector).shutdownServiceAndTelemetry(0xc000cbfa88, {0x103d90988?, 0xc0000580c0?})
go.opentelemetry.io/[email protected]/service/collector.go:235 +0x36
go.opentelemetry.io/collector/service.(*Collector).setupConfigurationComponents(0xc000cbfa88, {0x103d90988, 0xc0000580c0})
go.opentelemetry.io/[email protected]/service/collector.go:156 +0x286
go.opentelemetry.io/collector/service.(*Collector).Run(0xc000cbfa88, {0x103d90988, 0xc0000580c0})
go.opentelemetry.io/[email protected]/service/collector.go:165 +0x46
go.opentelemetry.io/collector/service.NewCommand.func1(0xc000158c00, {0x1036c9ce6?, 0x2?, 0x2?})
go.opentelemetry.io/[email protected]/service/command.go:53 +0x479
github.com/spf13/cobra.(*Command).execute(0xc000158c00, {0xc0000521c0, 0x2, 0x2})
github.com/spf13/[email protected]/command.go:916 +0x862
github.com/spf13/cobra.(*Command).ExecuteC(0xc000158c00)
github.com/spf13/[email protected]/command.go:1044 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/[email protected]/command.go:968

[djaglowski]

@Dongqi-Guo, please open a new issue for the new error. Please include your configuration as well.