-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Client loses connection with Event Hub and does not recover #24224
Comments
Hi @MickeyMouseS, Here are more details on the changes included: /cc @conniey |
Thank you @joshfree for your prompt answer. We will definitely update the library and let you know if we encounter any issue. Additionally, it would be really helpful if you can clarify these points:
|
Hi @MickeyMouseS, from the SDK side, we're not sure why the service would initiate closure of the endpoints (link, session, connection) without any "error". I would suggest opening a eventhub support ticket so that service dev can analyze the service log specific to your subscription/resource and clarify why this is happening. That said, you're right that SDK should retry/recover when such closure happens. There has been a couple of reliability fixes since 5.5.0, so like Josh pointed, please update to the latest to get those fixes; we specifically had fixes related to the close path. There is also a fix for close merged last week that will be released in Oct mid. |
We have same issue on azure-messaging-eventhubs 5.10.0. It happens multiple times per day. |
Also happens in 5.10.0, same time for two process running on different consumer groups |
If you could provide some INFO/DEBUG logs +/- 5 minutes after you experience the issue, it would help us narrow this down or a way to reproduce your issue/environment. :( We have a few stress infrastructure runs that have been running for ~2 weeks and have not experienced the issue. |
Hey, if you reach out directly ***@***.***) we will
provide you with jfr.
We are running for over a year now and it was the first time to happen.
…On Fri, 14 Jan 2022, 22:25 Connie Yau, ***@***.***> wrote:
If you could provide some INFO/DEBUG logs +/- 5 minutes after you
experience the issue, it would help us narrow this down or a way to
reproduce your issue/environment. :(
We have a few stress infrastructure runs that have been running for ~2
weeks and have not experienced the issue.
—
Reply to this email directly, view it on GitHub
<#24224 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAB5XYHVJLSQD5KT425Z5OLUWCIF5ANCNFSM5ENIFTWA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you! |
Hi dear bot. I guess friendly reminder does not work as I am happy to
provide info but did not get in touch from you :).
…On Sat, 22 Jan 2022, 03:02 msftbot[bot], ***@***.***> wrote:
Hi, we're sending this friendly reminder because we haven't heard back
from you in a while. We need more information about this issue to help
address it. Please be sure to give us your input within the next *7 days*.
If we don't hear back from you within *14 days* of this comment the issue
will be automatically closed. Thank you!
—
Reply to this email directly, view it on GitHub
<#24224 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAB5XYCPHBOZOSC7OGTLFLDUXIF2PANCNFSM5ENIFTWA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you! |
Our offer to provide jfr is still open. That way you will have unique
opportunity to analyse issue as happened in production.
…On Sat, 29 Jan 2022, 21:02 msftbot[bot], ***@***.***> wrote:
Hi, we're sending this friendly reminder because we haven't heard back
from you in a while. We need more information about this issue to help
address it. Please be sure to give us your input within the next *7 days*.
If we don't hear back from you within *14 days* of this comment the issue
will be automatically closed. Thank you!
—
Reply to this email directly, view it on GitHub
<#24224 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAB5XYGB67JMIQYKZAZZSNTUYRBUFANCNFSM5ENIFTWA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you! |
Yup still up for it.
…On Sun, 6 Feb 2022, 15:02 msftbot[bot], ***@***.***> wrote:
Hi, we're sending this friendly reminder because we haven't heard back
from you in a while. We need more information about this issue to help
address it. Please be sure to give us your input within the next *7 days*.
If we don't hear back from you within *14 days* of this comment the issue
will be automatically closed. Thank you!
—
Reply to this email directly, view it on GitHub
<#24224 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAB5XYFJWM2325CPAL372QLUZZ5N3ANCNFSM5ENIFTWA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you! |
TrackingID#2201040050000678
…On Sun, 13 Feb 2022, 21:02 msftbot[bot], ***@***.***> wrote:
Hi, we're sending this friendly reminder because we haven't heard back
from you in a while. We need more information about this issue to help
address it. Please be sure to give us your input within the next *7 days*.
If we don't hear back from you within *14 days* of this comment the issue
will be automatically closed. Thank you!
—
Reply to this email directly, view it on GitHub
<#24224 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAB5XYDNVWTSUCTE7OPHWN3U3AE4JANCNFSM5ENIFTWA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you! |
Sure. All is in the ticket. I heard its being analyzed. How do you think
bot?
…On Tue, 22 Feb 2022, 15:03 msftbot[bot], ***@***.***> wrote:
Hi, we're sending this friendly reminder because we haven't heard back
from you in a while. We need more information about this issue to help
address it. Please be sure to give us your input within the next *7 days*.
If we don't hear back from you within *14 days* of this comment the issue
will be automatically closed. Thank you!
—
Reply to this email directly, view it on GitHub
<#24224 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAB5XYHADYIYQCIGHCEL4B3U4OJSJANCNFSM5ENIFTWA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Hey @gondar, apologies for the bot bother, it doesn't seem to automatically take off the label. I was unable to catch your email because it was ***'d out. I looked into the issue "Tracking#2201040050000678" and the issue I have access to is a Service Bus related issue about overexcessive logging... I don't think that's the one you're referring to here. Since some automation censored out your email, you can reach out to me. My email is conniey [at] microsoft.com |
Hi all, Finally this issue has been resolved with the last library updates. Now the clients are able to recover AMQP communication after a disconnection event. The issue is resolved since version 5.11.1 (https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/eventhubs/azure-messaging-eventhubs/CHANGELOG.md#5111-2022-03-17). The "random" disconnection events are due to maintenance reasons in Azure Event Hub service. |
Describe the bug
Suddenly, after a period of activity, a consumer client connected to the Event Hub loses connection and never recovers, even though keeps on running.
Exception or Stack Trace
To Reproduce
It looks a random issue after a period of activity for some hours or days. It would be a big help to know how to produce that the event hub decides to closes the link, session and connection.
Code Snippet
It is the normal connection from examples:
Expected behavior
I would expect not to receive such events from the Event Hub (close of link, session and connection) and, if received, the client should recover opening a new connection. Or at least being able to manage this error. I would like also to have a description telling why the Event Hub decides to close the connection.
Setup (please complete the following information):
Additional context
It is actually the built-in Event Hub for the IoT Hub service.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
The text was updated successfully, but these errors were encountered: