Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUESTION] Proper errors handling #1866

Closed
kpeu3i opened this issue Dec 28, 2020 · 2 comments
Closed

[QUESTION] Proper errors handling #1866

kpeu3i opened this issue Dec 28, 2020 · 2 comments
Labels
documentation needs-investigation Issues that require followup from maintainers stale Issues and pull requests without any recent activity

Comments

@kpeu3i
Copy link

kpeu3i commented Dec 28, 2020

Versions

Sarama Kafka Go
v1.27.0 2+ 1.14.7

Configuration

We use SAAS kafka installation https://www.confluent.io/

config := sarama.NewConfig()
config.Version = sarama.V2_0_0_0

config.Net.SASL.Enable = t.config.SASL.Enabled
config.Net.SASL.User = t.config.SASL.User
config.Net.SASL.Password = t.config.SASL.Password
config.Net.SASL.Mechanism = sarama.SASLMechanism(t.config.SASL.Mechanism)
config.Net.TLS.Enable = t.config.TLS.Enabled

config.Metadata.Retry.Max = 10
config.Metadata.Retry.Backoff = time.Second

config.Consumer.MaxWaitTime = 1000 * time.Millisecond
config.Consumer.Return.Errors = true
config.Consumer.Group.Rebalance.Retry.Backoff = 5 * time.Second
config.Consumer.Group.Rebalance.Retry.Max = 10
// Possible fix for "The provided member is not known in the current generation" error.
// See https://github.com/Shopify/sarama/issues/1310#issuecomment-483768308
config.Consumer.Group.Session.Timeout = 20 * time.Second
config.Consumer.Group.Heartbeat.Interval = 6 * time.Second
config.Consumer.MaxProcessingTime = 500 * time.Millisecond

Can somebody clarify how to properly handle errors received from the consumer errors channel? Currently, it panics when an error occurred in the channel and this leads to the restart of an application.

go func() {
	for {
		select {
		case err := <-consumer.Errors():
			if err != nil {
				panic(err)
			}
		case <-t.quit:
			return
		}
	}
}()

Is possible to skip errors and continue listening to messages?

go func() {
	for {
		select {
		case err := <-consumer.Errors():
			if err != nil {
				log.Println(err)
			}
		case <-t.quit:
			return
		}
	}
}()

Why I am asking is because we see a lot of errors such:

kafka: error while consuming topic_name/partition: read tcp IP_1->IP_2: i/o timeout
kafka: error while consuming topic_name/partition: write tcp IP_1->IP_2: i/o timeout
@github-actions

This comment was marked as outdated.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Aug 25, 2023
@dnwe dnwe added documentation needs-investigation Issues that require followup from maintainers and removed stale Issues and pull requests without any recent activity labels Aug 25, 2023
Copy link

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur.
Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Nov 26, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Dec 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation needs-investigation Issues that require followup from maintainers stale Issues and pull requests without any recent activity
Projects
None yet
Development

No branches or pull requests

2 participants