kafka input stalls on high lag

[robertmarsal]

Since 4.9.1 (specifically this commit: 349ba19) when one of our Kafka topics has a high lag (more than 2000 pending messages) Benthos stops consuming without returning any errors and just stalls there.

It seems like the downgrade of the sarama library introduces this bug as previous versions before the rollback can handle this scenario just fine.

Looks like you might be waiting on IBM/sarama#2358 to be resolved to upgrade but that seems unlikely given how long has past. Would it be possible to upgrade to the latest version? Thanks

[robertmarsal]

We've updated to the latest version of sarama however it doesn't seem to fix this issues consistently so there might be other changes we are missing 🤔

[Jeffail]

Hey @robertmarsal, have you given the kafka_franz input a try? My medium-term plan is to deprecate kafka for it.

[robertmarsal]

Hi @Jeffail we looked into it and the only thing stopping us at the moment is the lack of extract_tracing_map in kafka_franz without it all our traces will get disconnected. Happy to add that if I could get some pointers 🤔

[jeongukjae]

Add a PR to add extract_tracing_map in kakfa_franz :)
#2836