Error while sending traces using OpenSearch exporter

[AmythD]

Component(s)

exporter/opensearch

Describe the issue you're reporting

Hello,

I am trying to send otel format application traces through an opentelemetry collector to an OpenSearch backend. I observed below errors on the otel collector logs and don't seem to be getting any traces on the opensearch side yet. I did not see any errors on the OpenSearch logs itself. Could anyone please help me with this?

2024-02-05T16:35:41.140Z warn [email protected]/batch_processor.go:258 Sender failed {"kind": "processor", "name": "batch", "pipeline": "traces", "error": "not retryable error: Permanent error: {\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse\",\"caused_by\":{\"type\":\"class_cast_exception\",\"reason\":\"class org.opensearch.index.mapper.KeywordFieldMapper cannot be cast to class org.opensearch.index.mapper.ObjectMapper (org.opensearch.index.mapper.KeywordFieldMapper and org.opensearch.index.mapper.ObjectMapper are in unnamed module of loader 'app')\",\"caused_by\":null}}"}

receivers:
   otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:

exporters:
  opensearch/trace:
    http:
      endpoint: http://X.X.X.X:9200
      tls:
        insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [opensearch/trace]
      processors: [batch]

Kindly excuse any incorrect indendation during copy paste of the config.

I'm on otel-contrib version 0.93.0 OpenSearch version 2.11.1

[github-actions]

Pinging code owners:

• exporter/opensearch: @Aneurysm9 @MitchellGale @MaxKsyunz @YANG-DB

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[MaxKsyunz]

This is a known issue.

OpenSearch ships with an index template that is used for indices with ss4o- prefix. This template fails to process the document it receives from otel exporter and fails insertion.

If you point exporter to an index with a different name, it should be able to index traces.

[chantzlarge]

@MaxKsyunz I've tried using the dataset and namespace params to change to target index but no luck... is this what you mean by:

If you point exporter to an index with a different name, it should be able to index traces.

[rodrigoserracoelho]

It also fails for me, even with a different index name:

opensearch/traces:
    dataset: other
    http:

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/opensearch: @Aneurysm9 @MitchellGale @MaxKsyunz @YANG-DB

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[PandeoF1]

Same issue

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/opensearch: @Aneurysm9 @MitchellGale @MaxKsyunz @YANG-DB

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[LucasGranela]

same error here

[korotkih2000]

Same problrm

[vamsi-amazon]

@korotkih2000 , @LucasGranela , @PandeoF1 , @rodrigoserracoelho
Could you all also provide the opensearch versions and otel versions that you guys are working on?