Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ungraceful shutdown of mysql-router-exporter can lead to connections in TIME_WAIT; missing graceful shutdown of server #59

Closed
shayancanonical opened this issue Apr 3, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@shayancanonical
Copy link

Issue

The exporter starts a server on listen-port using http.ListenAndServe. However, there is no attempt at a graceful shutdown when a SIGTERM is received (see this post for some ideas of how a SIGTERM can be reacted with a server.Shutdown())

We have an issue in our project where we use this project (mysqlrouter_exporter). As the trace in the issue indicates, we run into bind address in use error intermittently when we start exporter, stop it and then restart it. Further digging revealed that when we stop exporter, sometimes, due to an ungraceful shutdown, we have connections in TIME_WAIT:

root@mysql-router-k8s-0:/# netstat -nat  | grep 49152
tcp        0      0 10.1.241.229:49152      10.1.241.230:3306       TIME_WAIT

It is exactly under these conditions that we run into the bind address in use error. When the

Suggested Solution

To gracefully shutdown the server as indicated by the above linked post when a SIGTERM is received

Potential followups

I will attempt to create a PR with a proposed solution for this issue

@rluisr rluisr added the bug Something isn't working label Apr 3, 2024
@rluisr
Copy link
Owner

rluisr commented Apr 3, 2024

@shayancanonical Thank you for reporting.

I released a new version v5.0.1. If the problem has not been resolved, re-open it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants