[kafkareceiver] support session timeout configs (and maybe more) in kafka receiver. #28630
Comments
ben-childs-docusign
added
enhancement
|
Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
|
Hi @ben-childs-docusign, thanks for raising this request! |
|
Pinging code owners for receiver/kafka: @pavolloffay @MovieStoreGuy. See Adding Labels via Comments if you do not have permissions to add labels yourself. |
|
@VihasMakwana I can potentially contribute a change it just might take me a few weeks. My local change just hard codes the setting it doesn't fully connect the dots on the config. |
|
@ben-childs-docusign I've linked a PR that adds these settings |
|
This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
|
This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
|
This issue has been closed as inactive because it has been stale for 120 days with no activity. |
|
Can we reopen this? I have an active pr |
…agement facilities (#33082) Previous PR was closed due to inactivity waiting for review by maintainer: #32393 **Description:** To address #28630 i've wired in the session timeout and heartbeat interval settings. I would have exposed max.poll.interval as well but it appears Sarama doesn't make that setting available. It has a similar setting called `MaxProcessingTime` that we could implement if desired but behavior is different. Something to note is that Kafka recommends a 45s session timeout for consumers but Samara actually defaults to 10s so I've also defaulted to 10s to keep behavior consistent. **Link to tracking Issue:** #28630 **Testing:** I started the OTel collector with the following configuration and verified via debugger that the settings were applied to the Sarama client. ``` receivers: kafka: topic: test session_timeout: 15s heartbeat_interval: 5s exporters: debug: service: pipelines: logs: # a pipeline of “traces” type receivers: [kafka] exporters: [debug] ``` I then added a test to `config_test` that tests the default values and tests providing the values via config. **Documentation:** <Describe the documentation added.> I modified the readme to reference the new settings. --------- Co-authored-by: Sean Marciniak <[email protected]>
Component(s)
receiver/kafka
Is your feature request related to a problem? Please describe.
We are trying to configure our kafka receiver based on the azure event hub recommendations but some of the configuration values are not available. We were seeing unecessary consumer failovers in our collector. The default session timeout is 10s but eventhub recommends a timeout of 30 seconds for event hub consumers.
These are the recommended settings from Azure: https://learn.microsoft.com/en-us/azure/event-hubs/apache-kafka-configurations
Describe the solution you'd like
Add consumer.group.session.timeout config. E,g,
Describe alternatives you've considered
N/A
Additional context
No response
The text was updated successfully, but these errors were encountered: