Out of ordering of requests to opensearch

[Ramkumar-byte]

Hi Team,

We are seeing an out of order problem for the requests to opensearch when using index.write.method=upsert.

However we don't see this issue when we set index.write.method=insert.

Below is our connector config

connector.class=io.aiven.kafka.connect.opensearch.OpensearchSinkConnector
behavior.on.null.values=delete
transforms.extractKey.field=id
index.write.method=insert
connection.password=
tasks.max=10
topics=dev.task
batch.size=10000
transforms=extractKey
key.ignore=false
max.in.flight.requests=5
key.ignore.id.strategy=none
transforms.extractKey.type=org.apache.kafka.connect.transforms.ExtractField$Key
schema.ignore=true
key.converter.schemas.enable=false
flush.timeout.ms=300000
value.converter.schemas.enable=false
value.converter=org.apache.kafka.connect.json.JsonConverter
connection.url=...
key.converter=org.apache.kafka.connect.json.JsonConverter
behavior.on.version.conflict=warn
schemas.enable=false

[qlecorre]

Hey @Ramkumar-byte, thanks for reporting this issue with the sink connector. Is it still happening for you?

[nj4x]

FYI: We've experienced same issue. Apparently it can be fixed by setting max.in.flight.requests=1