Tests failed: System.Net.Sockets.Tests.DnsEndPointTest / *

[ghost]

Type of failures

Affected tests:

• Socket_StaticConnectAsync_HostNotFound
• Socket_StaticConnectAsync_Success
• Socket_StaticConnectAsync_ConnectionRefused
• Socket_ConnectAsyncDnsEndPoint_HostNotFound
• Socket_ConnectAsyncDnsEndPoint_Success (CI only)
• Socket_ConnectAsyncDnsEndPoint_SetSocketProperties_Success (CI only)

Timeout

Timed out while waiting for connection
Expected: True
Actual:   False
at System.Net.Sockets.Tests.DnsEndPointTest.Socket_StaticConnectAsync_Success(SocketImplementationType type)

OR

at System.Net.Sockets.Tests.DnsEndPointTest.Socket_StaticConnectAsync_ConnectionRefused()

OR

at System.Net.Sockets.Tests.DnsEndPointTest.Socket_StaticConnectAsync_HostNotFound() in /root/corefx-1213655/src/System.Net.Sockets/tests/FunctionalTests/DnsEndPointTest.cs:line 373

Assert failure

Test Socket_StaticConnectAsync_Success:

Assert.True() Failure
Expected: True
Actual:   False
at System.Net.Sockets.Tests.DnsEndPointTest.Socket_StaticConnectAsync_Success(SocketImplementationType type) in /root/corefx-1134004/src/System.Net.Sockets/tests/FunctionalTests/DnsEndPointTest.cs:line 312

History of failures

Day	Build	OS	Failure	Test
2017
5/31-7/25	N/A	Linux only	Timeout	51x Socket_StaticConnectAsync_HostNotFound 4x Socket_ConnectAsyncDnsEndPoint_HostNotFound, 1x Socket_StaticConnectAsync_Success
8/23	20170823.05	Win10	Timeout	Socket_StaticConnectAsync_Success
8/24	20170824.03	Win8.1	Timeout	Socket_StaticConnectAsync_ConnectionRefused
9/10	20170910.02	Win10	Timeout	Socket_StaticConnectAsync_Success
9/14	20170914.01	Win7	Timeout	Socket_StaticConnectAsync_Success
11/13	20171113.01	Centos73	Assert failure	Socket_StaticConnectAsync_Success
12/3	20171203.01	RedHat73	Assert failure	Socket_StaticConnectAsync_Success
12/11	20171211.05	Fedora25	Timeout	Socket_StaticConnectAsync_HostNotFound
12/23	20171223.02	OSX10.12	Timeout	Socket_StaticConnectAsync_Success - link
1/5	20180105.01	Win7	Timeout	Socket_ConnectAsyncDnsEndPoint_HostNotFound - link
1/5	20180105.01	Win7	Timeout	Socket_StaticConnectAsync_HostNotFound - link
1/9	20180109.01	Win7	Timeout	Socket_StaticConnectAsync_ConnectionRefused - link
3/5	20180305.02	Fedora26	Assert failure	Socket_StaticConnectAsync_Success
3/6	20180306.01	Win7	Timeout	Socket_StaticConnectAsync_Success
3/6	20180306.01	Win7	Timeout	Socket_StaticConnectAsync_Success
3/8	20180308.06	Fedora26	Timeout	Socket_StaticConnectAsync_HostNotFound
3/21	20180321.05	Win7	Timeout	Socket_StaticConnectAsync_Success

Original report

Opened on behalf of @JiayiLi1

The test System.Net.Sockets.Tests.DnsEndPointTest/Socket_StaticConnectAsync_HostNotFound has failed.

Timed out while waiting for connection
    Expected: True
    Actual:   False
at System.Net.Sockets.Tests.DnsEndPointTest.Socket_StaticConnectAsync_HostNotFound() in /root/corefx/src/System.Net.Sockets/tests/FunctionalTests/DnsEndPointTest.cs:line 373

Build : Master - 20170524.01 (Portable Core Tests)
Failing configurations:

• suse.422.amd64-x64
  • Release

Detail: https://mc.dot.net/#/product/netcore/master/source/official~2Fcorefx~2Fmaster~2F/type/test~2Ffunctional~2Fportable~2Fcli~2F/build/20170524.01/workItem/System.Net.Sockets.Tests/analysis/xunit/System.Net.Sockets.Tests.DnsEndPointTest~2FSocket_StaticConnectAsync_HostNotFound

[davidsh]

This test uses a loopback socket server. So, not sure why it would fail with timeout.

[wfurt]

Last failure in Helix is from 2017-07-25. It is interesting that CI has disproportionally more failures.

[karelz]

History of test failure updated in top post

[wfurt]

In last 90 days we failed:
Socket_StaticConnectAsync_HostNotFound 3x
Socket_StaticConnectAsync_Success 2x
Socket_ConnectAsyncDnsEndPoint_Success 3x

seems like on-off failures across multiple OSes.
If we care enough we may benefit from retries as neither one looks like massive failure.
We should also instrument tests to dump useful information - SocketAsyncEventArgs.SocketError, perhaps others.
It may be worth of exploring if slow or unresponsive DNS server would lead to same test failure.

[karelz]

Failures during 2021/3/12-8/25 (incl. PRs):

Date	Build / PR	OS	Test
2021			Excluding release/6.0* branches
3/21	Official run	Alpine.312.Amd64.Open	Socket_StaticConnectAsync_Success
4/22	Official run	Alpine.312.Amd64.Open	Socket_StaticConnectAsync_Success
4/30	Official run	Windows.10.Amd64.Server19H1.Open	Socket_ConnectAsyncDnsEndPoint_Success
5/2	Official run	Fedora.30.Amd64.Open	Socket_StaticConnectAsync_Success
5/12	Official run	Alpine.312.Arm64.Open - arm64	Socket_StaticConnectAsync_Success
5/12	Official run	OSX.1014.Amd64.Open	Socket_StaticConnectAsync_ConnectionRefused
6/11	Official run	Fedora.30.Amd64.Open	Socket_StaticConnectAsync_Success
6/20	Official run	Debian.9.Amd64.Open	Socket_StaticConnectAsync_Success
6/27	Official run	Fedora.32.Amd64.Open	Socket_StaticConnectAsync_Success
7/8	Official run	Debian.9.Amd64.Open	Socket_StaticConnectAsync_Success
7/16	Official run	Ubuntu.1804.ArmArch.Open - arm64	Socket_StaticConnectAsync_Success
7/16	Official run	Windows.Nano.1809.Amd64.Open	Socket_ConnectAsyncDnsEndPoint_HostNotFound
7/22	Official run	Debian.9.Arm32.Open - arm32	Socket_StaticConnectAsync_Success
8/23	PR#57745	OSX.1015.Amd64.Open - Mono	Socket_StaticConnectAsync_Success

[wfurt]

triage: no recent failures. Part of the DNS failures are infrastructure issues. If we see new failures we should open new issue to start clean. Since this depends on external services it if very difficult to make it 100% reliable.