Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.
/ corefx Public archive

Fix for 2nd Connection hangs in Mirroring Environment #24181

Merged
merged 7 commits into from
Sep 21, 2017

Conversation

geleems
Copy link

@geleems geleems commented Sep 20, 2017

This is a fix for Issue: https://github.com/dotnet/corefx/issues/24148
Customer reported a bug for hanging connection to mirrored SQL Server. When customer creates and opens 2 connections one after another to mirrored server with connection timeout set to 0, the 2nd connection hangs infinitely.

}
else
{
//thread.Abort() is not implemented yet in CoreFx.
Copy link
Author

@geleems geleems Sep 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Async task with timeout does not work in this case. Async task is basically run by child thread in background, and main thread uses CancellationTokenSource.Cancel() (https://msdn.microsoft.com/en-us/library/dd321955.aspx) to request cancelling the task when it is timed out. In CancellationTokenSource.Cancel(), main thread changes the cancellation token flag to true. And the child thread watches the flag, stops its work, and joins back to main thread. This only works when child thread is responsive and able to watch the cancellation token flag. In my case, the child thread gets stuck in opening connection procedure as spinning infinitely, and is no longer responsive or available to accept the flag.
I will use thread.Interrupt() for now since thread.Abort() is not implemented yet in CoreFx. Since the thread is interrupted, it does not spin anymore, just does not release the memory it occupies, which does not affect the performance of next unit tests.


stopWatch.Start();
childThread.Start();
while (!worker.IsDone && stopWatch.ElapsedMilliseconds <= 10000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use Thread.Sleep() for a few milliseconds if the worker hasn't completed?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because child thread could finish its job earlier than the number of milliseconds the main thread sleeps for. Main thread does not need to wait extra time after child finishes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10000 ms wait is arbitrary. shouldn't we wait for the connection timeout period?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using ManualResetEvent to signal when the thread completes, and then waiting on that for 10 seconds instead?
https://msdn.microsoft.com/en-us/library/system.threading.manualresetevent(v=vs.110).aspx

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@saurabh500 the connection timeout for testing the scenario of the bug (https://github.com/dotnet/corefx/issues/24148) is 0.

Copy link
Author

@geleems geleems Sep 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@corivera Is this what you meant?

  • Main thread is blocked by ManualResetEvent.WaitOne(10) after starting child thread.
  • Child thread completes its task, and calls ManualResetEvent.Set()
  • Main thread is unblocked, and finishes the application.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reset() doesn't block the thread, it just sets the Event state to nonsignaled. You need to call WaitOne() on the Event to actually wait on it. Instead of setting IsDone=true, you just call ManualResetEvent.Set(), which will unblock waiting thread.

Also, where are you setting timeout=0 in the test? The default connection timeout is 15 seconds.

Copy link
Author

@geleems geleems Sep 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it does the same thing anyway. The only difference is the main thread spins until the child thread finishes or be timed out.
And, this is a manual test. Connection string including timeout is provided manually by user. I will put a line to force to set timeout 0.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to you. It just seems like ManualResetEvent signaling does exactly what you want without needing any of the busy spinning (for up to 10 seconds) in between.

{
//thread.Abort() is not implemented yet in CoreFx.
childThread.Interrupt();
throw new Exception();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a descriptive message to this exception?

string failoverPartnerName;
bool isMirroring = GetMirroringInfo(connectionString, out mirroringState, out failoverPartnerName);
List<SqlConnection> list = new List<SqlConnection>();
if (isMirroring && mirroringState == 4 && !string.IsNullOrEmpty(failoverPartnerName))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any enum in SqlClient which better describes that mirroring state number?

Copy link
Member

@corivera corivera Sep 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If not, can you create a local variable which describes what the state number means?

int mirroringState;
string failoverPartnerName;
bool isMirroring = GetMirroringInfo(connectionString, out mirroringState, out failoverPartnerName);
List<SqlConnection> list = new List<SqlConnection>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't look like this list is used anywhere.

SqlConnectionStringBuilder existingConnStrBuilder = new SqlConnectionStringBuilder(connectionString);
SqlConnectionStringBuilder newConnStrBuilder = new SqlConnectionStringBuilder();
newConnStrBuilder.DataSource = existingConnStrBuilder.DataSource;
if (!string.IsNullOrEmpty(existingConnStrBuilder.UserID))
Copy link
Member

@corivera corivera Sep 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to manually check all the state if you just copy the same values anyway? Wouldn't it be better just to create a builder from the same source and clear the things you explicitly don't want?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you clear the unwanted options out?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mre.Reset();
mre.WaitOne(10000);

if (worker.IsDone)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can check the Boolean result of WaitOne() here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool!

{
public static class ConnectionOnMirroringTest
{
private static ManualResetEvent mre = new ManualResetEvent(false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you rename this to something more descriptive? Maybe "workerCompletedEvent", or something similar.

string failoverPartnerName;
bool isMirroring = GetMirroringInfo(connectionString, out mirroringStateDesc, out failoverPartnerName);
bool isSynchronized = "SYNCHRONIZED".Equals(mirroringStateDesc, StringComparison.InvariantCultureIgnoreCase);
List<SqlConnection> list = new List<SqlConnection>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This list doesn't get used anywhere.

Thread childThread = new Thread(() => worker.TestMultipleConnection());
childThread.Start();

mre.Reset();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need to call reset, since the initial state is set to nonsignaled by passing false to the constructor (as you've done above).

@geleems geleems merged commit dd48b05 into dotnet:master Sep 21, 2017
beniamin-airapetian added a commit to beniamin-airapetian/corefx that referenced this pull request Sep 23, 2017
* Microsoft.ServiceModel.Syndication skeleton project

* Adding the existing classes of SyndicationFeed from .net fx

* Added the needed references to get the code to compile

* Changed some namespaces

* Fixed errors when reading feeds and replaced some buffers

* Cleaning code for PR

* Deleted some unused files

* Added the posibility that the user creates his own date parser when reading a feed from outside, also as part of our default date parser, if it can't parse the date it will just assign a default date to avoid the crash

* Added correct testnames and Copyright

* Initial changes to add custom parsers

* Added more delegates as parsers

* Added custom parser delegates

* Save changes

* Initial SyndicationFeed

* Removed the dependence of the old SR class

* Cleaned the code, deleted Diagnostics, fixed throwing resources

* Cleaned code from most of the unnecesary comments

* Formatted the code with CodeFormatter Tool

* Moved the call of itemParser to be called with each item

* Test using custom parsers

* Fixed image issue where image tag was using original title and url, for RSS formatting
Image issue fixed and added some tests

* Test clase jorge

* Save changes with Jorge cass

* Initial Jorge

* saving changes

* Save changes

* Fixed image and items issue

* Fixed disjoint items in atom

* Run codeFormatter

* Adding parsing to optional elements int the spec
added unittesting

* Added Icon parsing to Atom10 feedFormatter. Added unit test

* Adding Async Writing
RssFormater writes new optional spec elements

* Added Icon writing on Arom Writer

* Fixed some warnings

* Improved custom parsing for RSS

* Added custom parsing for Atom feed formatter, added test

* added nameof() to all exceptions when needed.

* Adding Extension Methods for XmlReader

* Fixing code for PR

* Fixing code for PR

* Added check for skip days to allow only accepted days.

* Improved flexibility for Default dateparser

* Add wrong dates example

* Fixing code review

* Fixed warnings on some unawaited methods.

* Added async extension methods for XmlReader

* Add XmlWriterWrapper method extensions

* Changed ReadCategoryFromAsync to return a SyndicationCategory

* Fixed sync method exposed GetReader.

* Edited XmlReaderWrapper, moved methods to extension methods.

* Removed unnecesary variables from Wrappers.

* Fixed bug from ServiceModel.Syndication

* Make BigInteger ctor opt path for == int.MinValue reachable.

The BigInteger(ReadOnlySpan<byte> value) ctor contains an optimised
path for value being larger than 4 bytes and the result being equal to
int.MinValue.

However this path is inside a test that precludes that value, so can
never be reached.

Restructure so that the path is reachable.

* Fix ServiceModel.SyndicationFeed project dependencies
- Change M.SM.SyndicationFeed to be a .NET Standard 2.0 library
- Change tests to use official .NET Core 2.0 release from preview

* Add btrfs and other missing file system types (dotnet#24102)

* Uppercase

* Add more filesystems from stat.c. Change to stat names where possible

* Add some more local friendly names

* Add some more remote file types from stat.c

* Add entry to switch present in enum

* Build errors

* Remove Fixed entries that should be RAM

* comment

* Change case of 0X to 0x

* Move cramfs to Fixed

* ConcurrentQueue 128-byte cache line (dotnet#22724)

ConcurrentQueue 128-byte cache line

* Add warning by default in SGEN (dotnet#24054)

* Output warning by default if run the tool directly without /quiet parameter.

* add quiet parameter in the command.

* fix parameter error.

* Update the warning.

* Add the target to copy the serializer to publish folder. (dotnet#24096)

* Wrap cert callback leaked exceptions in WinHttpHandler (dotnet#24107)

Similar to the fix done for CurlHandler (dotnet#21938), fix WinHttpHandler so
that leaked exceptions from the user-provided certificate callback are
properly wrapped.

The ManagedHandler still needs to be fixed since it is not wrapping
properly.

Contributes to #21904

* Ensure ProcessModule for main executable is first in modules list (dotnet#24106)

* Disable NegotiateStreamTest fixture for distros without working Kerberos (dotnet#24098)

Disable NegotiateStreamTest fixture entirely because setup-kdc.sh is broken on some distros

* Expose and add tests for Guid.ctor(Span)/TryWriteBytes/TryFormat

* Address remaining PR feedback

* #24112 Replaced documentation summary of TryPop with text similar to TryDequeue. (dotnet#24113)

* Wrap exceptions from ManagedHandler's server validation callback (dotnet#24111)

To match other handlers' behaviors.

* Fix libgdiplus function loading on OSX.

* Prevent ServiceControllerTests from dispose when already disposed (dotnet#24042)

* Disable ServiceProcessTest that has been failing on CI and official builds

* Prevent dispose from been called when already disposed

* Add logging to repro if RemoveService is been called twice on the same ServiceController

* Data-annotations length fix (dotnet#24101)

* Updated in MinLengthAttribute and MaxLengthAttribute to support ICollection<T>

* Added tests

* Fixed typo

* Trying to address two failing checks:
- Linux x64 Release Build
- UWP CoreCLR x64 Debug Build

* Implemented changes requested in review
- Extracted Count checking to an external helper to obey DRY
- Removed dependency of ICollection<T> and changed to simple reflection Count property lookup

* Added requested tests

* Added catch for MissingMetadataException.

* Extracted code from try-catch.

* Added comment as requested.

* Typo correction

* Remove System.Drawing.Common from the netfx compat package (temporary).

* Updating corefx to use new roslyn compiler (dotnet#24076)

* Updating corefx to use new roslyn compiler

* Updating to new version of the compiler and use the switch so tests pass on desktop

* Fix System.Reflection.Metadata tests

* Stop running Math.Clamp tests on UAP as API is not there (dotnet#24125)

* Stop running Math.Clamp tests on UAP as API is not there

* Disable only for AOT

* Fix instantiating PrincipleContext  with Domain type (dotnet#24122)

* Fix instantiating PrincipleContext  with Domain type

* Enhancement

* Disable Test_Write_Metric_eventListener on UWP. (dotnet#24127)

* Revert "Remove System.Drawing.Common from the netfx compat package (temporary)."

This reverts commit f6b0fbd.

* Update BuildTools, CoreClr, CoreFx, CoreSetup, Standard to prerelease-02019-01, preview1-25718-02, preview1-25718-03, preview1-25717-02, preview1-25718-01, respectively (dotnet#24075)

* Fixed compile warning/error on FreeBSD (dotnet#24141)

* Marking {ReadOnly}Span as readonly structs (dotnet#23908)

* Marking {ReadOnly}Span as readonly structs, fixing issue #23809

* Adding readonly attributes to reference assemblies.

* Using "readonly ref" keyword instead of attributes.

* Adding a LangVersion 7.2 property

* System.Drawing: Throw ArgumentNullException on Unix as well (dotnet#24140)

* Throw ArgumentNullException when stream is null

* Throw ArgumentNullException when stream is null

* Remove invalid NameResolution tests (dotnet#24147)

The Dns_GetHostEntryAsync_* are fundamentially invalid because it's
possible for the broadcast address, 255.255.255.255, to have an DNS
mapping via manually modifying the hosts file.  This was actually
happening on Mac systems as well as an virtual environment running on
top of that (i.e. Windows on Parallels).

Ref:
https://github.com/dotnet/corefx/issues/23992#issuecomment-330250642

Contributes to #23992

* Bump system.drawing.common.testdata to 1.0.6

* Update BuildTools to prerelease-02019-02 (dotnet#24146)

* Fix path to test data

* Fix whitespace

* Add GraphicsTests based on Mono's test suite.

* Consolidate more code in the "System.Drawing" namespace.

* Remove all remaining Win32 codepaths from the mono codebase. All of this
  code now implicitly assumes that it will be run on a Unix platform.
* Consolidate the rest of the gdipFunctions.cs file into Gdip.cs and
  GdipNative.Unix.cs
* Consolidate the GraphicsUnit and ImageType enumerations -- they were
  duplicated.
* Remove the mono Status enum and use the Windows constants instead in all
  Unix code.
* Move all files into the regular directory structure. Suffix them with
  ".Unix" and ".Windows" when there are collisions.

* Tiny bit of code cleanup

* Add conditionals for recent versions of mono

* Remove duplicate tests.

* Fix multiplying TextureBrush with a disposed matrix on Unix (dotnet#24109)

* Fix multiplying TextureBrush with a disposed matrix on Unix

* Enable another passing test

* Fix accidentally committed file

* Add an error code fixup to Bitmap.Unix.cs to match Windows.

* Bump System.Drawing.Common.TestData to 1.0.6 (dotnet#24149)

* Bump system.drawing.common.testdata to 1.0.6

* Fix path to test data

* Fix whitespace

* Cleanup - Add/simplify using statements of disposable resources (Graphics, Bitmap)

* Validate HatchStyle passed to HatchBrush ctor

* Delete accidentally duplicated HatchBrush tests in LinearGradientBrushTests

* Remove references to historical Mono bug IDs

* Renable some already passing tests

* Remove Thread.Sleep workaround

* Update BuildTools to prerelease-02020-01 (dotnet#24172)

* Add thread-local based switch to opt-in to ManagedHandler

* Address PR feedback

* PR feedback

* Fix memory map imports (dotnet#24176)

* Fix memory map imports

Imports lost the last error attribute. Add it back and change the results to be the more correct "bool". Tweak the usage based on the new return type.

#24159

* Move spin wait

* Remove unused FEATURE_RANDOMIZED_STRING_HASHING (dotnet#24178)

* Adding System.Data.Odbc package and including in metapackage

* Fix MultiplyTransform with a disposed brush

* Fix for 2nd Connection hangs in Mirroring Environment (dotnet#24181)

* Switch tests to use thread-local switch for ManagedHandler

And as a result re-enable parallelism of the test suite, which on my machine reduces the running time of the outerloop tests from 150s to 45s.

* Update CoreClr, CoreSetup, ProjectNTfs, ProjectNTfsTestILC, Standard to preview1-25720-03, preview1-25719-04, beta-25721-01, beta-25721-01, preview1-25721-01, respectively

* CoreFx #22406 Span based APIs - Text Reader Writer (dotnet#23786)

* [Drawing] Move remaining "Unix" files into regular directory structure

* As part of this, the "Windows" version of ImageFormat.cs is now being
  used everywhere. This file just contains a bunch of GUID's that are not
  platform-specific in any way.

* Change AsSpan to property Span and rename AsMemory to Memory

* Add Metafile and MetaHeader tests based on Mono's System.Drawing unit tests

* Update project file

* MetafileTests: Remove duplicates, and re-enable tests on Unix which are now working.
Delete duplicate MetaHeadertests

* Rationalize using statements

* Update project file

* Fix Metafile exception behavior

* Simplify tests, remove duplicate test

* Don't catch an ArgumentException just to get a NullReferenceException instead.

* Assert.False(Object.ReferenceEquals => Assert.NotSame

* PR feedback

* Remove duplicate tests, fix typo

* Remove X11 dependency for tests which are enabled on Unix.

* Workaround libgdiplus glitches

Don't test metafileHeader.Bounds when using libgdiplus.
Don't test metafile.GetBounds on Unix, force MetafileHeader.Bounds to return an empty rectangle when MetafileSize == 0
Don't validate graphicsUnit on Unix.

* Move Syndication to root/src

* Disable Build of S.SM.Syndication.

* Increase file descriptor limit in S.N.Http tests on macOS

* Expose/test String.Create span-based method (dotnet#23872)

* Expose/test String.Create span-based method

* Address PR feedback

* Update build to clang/llvm 3.9 (dotnet#24177)

Update scripts, docs and build pipeline docker images to clang/llvm/lldb 3.9

* Update ProjectNTfs, ProjectNTfsTestILC, Standard to beta-25722-00, beta-25722-00, preview1-25722-01, respectively (dotnet#24207)

* Remove the line that will copy the generated serializer to the pack. (dotnet#24199)

* Revert "Update build to clang/llvm 3.9 (dotnet#24177)"

This reverts commit 21e008a.

* Remove stale SetStateMachine call in test

* Ssl Stream Async Write  (dotnet#23715)

* Change from APM to Async/Await for write side

* Added nameof

* Reacting to review

* Reacting to review

* SSLStream : Fixed spelling mistake in file name (extra a) (dotnet#24221)

* Fixed spelling mistake in file name

* Fix csproj

* Update ProjectNTfs, ProjectNTfsTestILC to beta-25723-00, beta-25723-00, respectively (dotnet#24222)
@karelz karelz added this to the 2.1.0 milestone Oct 11, 2017
@geleems geleems deleted the hang branch November 2, 2017 19:56
picenka21 pushed a commit to picenka21/runtime that referenced this pull request Feb 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants