Fix Start ctx expiration making app unstoppable #1017
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This fixes uber-go#1015. When Start() is called, it passes StartTimeout to the relayer goroutine, which is monitoring the Stop signal, as well as the OS signal handler. It then exits the goroutine if the start context expires, which would naturally happen if the user uses the default setting (15s timeout) or any finite amount of timeout value. This causes the app to then be not responsive to any OS signals such as SIGTERM or SIGINT. This was a regression introduced in 1.19 release. To fix this, this commit simply removes the relayer goroutine from selecting on the start context being completed. Verified that all tests are still passing without any goroutine leak, except one test that was triggering a panic to test the panic handler. To prevent that one test from opting out every single test into goleak.VerifyNone(t) in every sub test, I pulled out that panic test into a separate test package so that we can continue to use goleak.VerifyNone() method in app_test.
Codecov Report
@@ Coverage Diff @@
## master #1017 +/- ##
==========================================
- Coverage 98.14% 98.14% -0.01%
==========================================
Files 39 39
Lines 1994 1992 -2
==========================================
- Hits 1957 1955 -2
Misses 29 29
Partials 8 8
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
Note to reviewers: will kick off internal Go monorepo build pinning it to this branch |
tchung1118
approved these changes
Jan 9, 2023
JacobOaks
approved these changes
Jan 9, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes #1015.
When Start() is called, it passes StartTimeout to the relayer goroutine, which is monitoring the Stop signal, as well as the OS signal handler.
It then exits the goroutine if the start context expires, which would naturally happen if the user uses the default setting (15s timeout) or any finite amount of timeout value.
This causes the app to then be not responsive to any OS signals such as SIGTERM or SIGINT.
This was a regression introduced in 1.19 release via #989 .
To fix this, this commit simply removes the relayer goroutine from selecting on the start context being completed.
Verified that all tests are still passing without any goroutine leak, except one test that was triggering a panic to test the panic handler.
To prevent that one test from opting out every single test into goleak.VerifyNone(t) in every sub test, I pulled out that panic test into a separate test package so that we can continue to use goleak.VerifyNone() method in app_test.