Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize host shutdown #249

Merged
merged 2 commits into from
Apr 20, 2023
Merged

Optimize host shutdown #249

merged 2 commits into from
Apr 20, 2023

Conversation

sebastianburckhardt
Copy link
Member

Currently, when the EventHubsTransport shuts down, it shuts down the client and closes the EH connection only after all the workers are shut down.

This can cause problems if the worker shut down hangs as observed in #245, because the EH client does not shut down, causing issues.

This PR modifies the shut down so that clients & connections are shut down in parallel with the workers. This fixes the problem described above, and also improves the speed of the shutdown process.

Copy link
Member

@davidmrdavid davidmrdavid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - just two suggestions for extra logging. Please consider adding them if you think the granularity will help with debugging.

Comment on lines +304 to +308
this.traceHelper.LogDebug("EventHubsTransport is stopping partition and loadmonitor hosts");
await Task.WhenAll(
this.StopPartitionHostAsync(),
this.StopLoadMonitorHostAsync());
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want a log indicating that the WhenAll completed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can skip that one since there is a log statement at the very end of each of the tasks.

Comment on lines 318 to 320
this.traceHelper.LogDebug("EventHubsTransport is stopping client");
await this.client.StopAsync();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want a log indicating the client stopped, not just that it began its stop operation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that makes sense.

@davidmrdavid
Copy link
Member

hmm, I see the CI is failing though. Is it transient or related to this change?

@sebastianburckhardt
Copy link
Member Author

The CI failures are likely not related. I have seen them a few times already; will work on that separately.

@sebastianburckhardt sebastianburckhardt merged commit 6090def into dev Apr 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants