Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "server: factor out MainCommon as a class, with a run method (… #2613

Merged
merged 1 commit into from
Feb 15, 2018

Conversation

htuch
Copy link
Member

@htuch htuch commented Feb 15, 2018

#2568)"

This reverts commit deffec6, which was
causing CI to flake today in TSAN runs. The proximate root cause is
the change in #2568, which appears not to correctly setup logging in
--mode=validate. This in turn leads to some log related races that I
don't fully grok.

Rolling back to unblock CI, we should make sure hotrestart_test passes
TSAN with high confidence before rolling forward again.

Risk Level: Low
Testing: bazel test //test/integration:hotrestart_test --runs_per_test=100 --config=clang-tsan -c dbg

Signed-off-by: Harvey Tuch [email protected]

…nvoyproxy#2568)"

This reverts commit deffec6, which was
causing CI to flake today in TSAN runs. The proximate root cause is
the change in envoyproxy#2568, which appears not to correctly setup logging in
--mode=validate. This in turn leads to some log related races that I
don't fully grok.

Rolling back to unblock CI, we should make sure hotrestart_test passes
TSAN with high confidence before rolling forward again.

Signed-off-by: Harvey Tuch <[email protected]>
@htuch
Copy link
Member Author

htuch commented Feb 15, 2018

@jmarantz @envoyproxy/maintainers Apologies for the rollback, but this will unblock CI. We should try and figure out what's up with --mode=validate after.

@jmarantz
Copy link
Contributor

Sorry about the regression! I wonder if this would've been easier to catch with the thread annotation you've now enabled?

@htuch
Copy link
Member Author

htuch commented Feb 15, 2018

@jmarantz Yeah, it will be interesting to understand the underlying race; I think somehow we were getting default logging initialization effects rather than a full initialization prior to threads launching (which were then logging). Some of the locking around logging could benefit from thread annotations.

@htuch htuch merged commit 034135f into envoyproxy:master Feb 15, 2018
@htuch htuch deleted the fix-tsan branch February 15, 2018 03:54
jmarantz added a commit to jmarantz/envoy that referenced this pull request Feb 15, 2018
dnoe pushed a commit that referenced this pull request Feb 15, 2018
…ation (#2623)

Description:
Reverts #2613 the revert of #2568
Fixed the underlying issue which was that logging was not initialized during validation, so it ran lockless. This caused intermittent tsan errors. Although #2619 makes the failure immediate and consistent by asserting that logging has been initialized prior to spawning any threads.

Risk Level: Medium -- the earlier #2568 caused intermittent tsan errors which we believe to be fixed, but #2568 was medium-risk in the first place.

Release Notes: N/A
Shikugawa pushed a commit to Shikugawa/envoy that referenced this pull request Mar 28, 2020
This can save ~1ms at batch size == 100.
jpsim pushed a commit that referenced this pull request Nov 28, 2022
Risk Level: low
Testing: unit tests
Docs Changes: n/a
Release Notes: yes
Part of #1594

Signed-off-by: Alyssa Wilk <[email protected]>
Signed-off-by: JP Simard <[email protected]>
jpsim pushed a commit that referenced this pull request Nov 29, 2022
Risk Level: low
Testing: unit tests
Docs Changes: n/a
Release Notes: yes
Part of #1594

Signed-off-by: Alyssa Wilk <[email protected]>
Signed-off-by: JP Simard <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants