Update CoreClr, CoreFx, External, ProjectNTfs, Standard to beta-25019-02, beta-25018-01, beta-25016-01, beta-25017-00, beta-25017-01, respectively (master)

[dotnet-bot]

No description provided.

[stephentoub]

@jkotas, we're seeing random failures and seg faults across various Unixes when updating to the new CoreCLR, all appear to be stemming from the System.Collections.Tests tests:
https://ci.dot.net/job/dotnet_corefx/job/master/job/centos7.1_release_prtest/3543/
https://ci.dot.net/job/dotnet_corefx/job/master/job/centos7.1_release_prtest/3538/
https://ci.dot.net/job/dotnet_corefx/job/master/job/osx_release_prtest/3460/
https://ci.dot.net/job/dotnet_corefx/job/master/job/osx_release_prtest/3452/
https://ci.dot.net/job/dotnet_corefx/job/master/job/portablelinux_release_prtest/804/
Any ideas what change in CoreCLR may have caused this?

[stephentoub]

cc: @danmosemsft

[danmoseley]

@stephentoub do you repro these locally? If not I can try.

[stephentoub]

@stephentoub do you repro these locally? If not I can try.

I haven't tried.

[danmoseley]

OK, will do.

[danmoseley]

Oop, I see it doesn't repro on Ubuntu, which is what I have (and indeed it doesn't).

[danmoseley]

@ianhays since it's collections -- do you have a mac to try this locally?

[ianhays]

Sure do. I'll see if I can repro it.

[AndyAyersMS]

This probably has an obvious answer, but how does one map from the nuget versions back to commit ranges?

[danmoseley]

The only way I know is

• Go to myget for this version https://dotnet.myget.org/feed/dotnet-core/package/nuget/runtime.win7-x64.Microsoft.NETCore.Runtime.CoreCLR/1.2.0-beta-25015-04 and click download
• Open the nupkg (zip) and the corelib in ilspy or similar. Eg., from runtimes\win7-x64\lib\netstandard1.0\
• Look for AssemblyInformationalVersion eg [assembly: AssemblyInformationalVersion("4.6.25015.04 built by: dlab-DDVSOWINAGE016. Commit Hash: 94d1639d5d0b969489abbb3ce78b11a72c4493c4")]

This is obviously painful, and you have to repeat it to get a range.

Perhaps @stephentoub or @ianhays have a better way - there really ought to be a better way.

[danmoseley]

In this case it gives me a range from 25014.04 to 24015.04 of
git log 9759c8d4c13155c64e707989106b36b88c044579..94d1639d5d0b969489abbb3ce78b11a72c4493c4

[danmoseley]

Or maybe @MattGal knows a better way.

[AndyAyersMS]

Looks like the 15-02 version passed (see #16168) , so perhaps focus on the range after that?

[stephentoub]

Looks like the 15-02 version passed (see #16168) , so perhaps focus on the range after that?

It didn't, actually. That accidentally got merged during the Jenkins outage when no CI legs were being run. I reverted it because I was then seeing this failure sporadically across the repo. Which itself suggests that instead of focusing on the range after that, you could likely focus on the range before that :)

[AndyAyersMS]

Ah -- so the last good version was indeed 25014-04...

[MattGal]

@danmosemsft I do not know a better way to do this than checking the packages themselves. The only other place I know that we really have this information is in the build definitions output themselves, which takes looking in a few different places on VSO to infer.

[ianhays]

Sure do. I'll see if I can repro it.

No luck yet. Ran them in a loop about 50 times on OSX and didn't get any failures from this PR.

[jkotas]

You may want to try checked build of CoreCLR; and if it still does not repro - also run it with GC stress on.

It is likely a regression from one of the recent JIT changes.

[AndyAyersMS]

I saw the older CI build fail on this on Windows and I was able to repro this locally after 4 tries:

 call C:\repos\corefx\bin\testhost\netcoreapp-Windows_NT-Release-x64\\dotnet.exe xunit.console.netcore.exe System.Coll
  ections.Tests.dll  -xml testResults.xml -notrait Benchmark=true -notrait category=nonnetcoreapp1.1tests  -notrait cat
  egory=OuterLoop -notrait category=failing -notrait category=nonwindowstests
  xUnit.net console test runner (64-bit .NET Core)
  Copyright (C) 2014 Outercurve Foundation.

  Discovering: System.Collections.Tests
  Discovered:  System.Collections.Tests
  Starting:    System.Collections.Tests
     System.Collections.Generic.Tests.ComparerTests.NullableOfIComparableComparisonsShouldTryToCallLeftHandCompareToFir
  st [FAIL]
        Assert.Equal() Failure
        Expected: 1
        Actual:   0
        Stack Trace:
           C:\repos\corefx\src\System.Collections\tests\Generic\Comparers\Comparer.Tests.cs(343,0): at System.Collectio
  ns.Generic.Tests.ComparerTests.NullableOfIComparableComparisonsShouldTryToCallLeftHandCompareToFirst()
  Finished:    System.Collections.Tests

Will see if I can pin this down.

[AndyAyersMS]

Guessing maybe the "update" job on CoreCLR has to go through first? Will keep an eye on it.

[stephentoub]

Thanks, Andy. Once a new coreclr build is available, this PR will be updated to it automatically. Looks like the last one was just before the revert.

[AndyAyersMS]

There are arm64 build failures in CoreCLR, might that block the update process?

[AndyAyersMS]

The 25019-01 build of CoreCLR just finished and should have the fix.

[danmoseley]

XML tests now. This is not showing up in the regular CI or Jenkins.

System.Xml.Tests.XmlWriterTests_EncodingFallback.AsyncXmlWriterConvertsSurrogatePairToEntity
System.Xml.Tests.XmlWriterTests_EncodingFallback.AsyncXmlWriterConvertsInvalidCharacterToEntity
System.Xml.Tests.XmlWriterTests_EncodingFallback.XmlWriterConvertsInvalidCharacterToEntity
System.Xml.Tests.XmlWriterTests_EncodingFallback.XmlWriterConvertsSurrogatePairToEntity

MESSAGE:
System.AggregateException : One or more errors occurred. (Unable to translate Unicode character \\u10000 at index 4 to specified code page.)\n---- System.Text.EncoderFallbackException : Unable to translate Unicode character \\u10000 at index 4 to specified code page.
+++++++++++++++++++
STACK TRACE:
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at System.Xml.Tests.XmlWriterTests_EncodingFallback.AsyncXmlWriterConvertsSurrogatePairToEntity() in /mnt/resource/j/workspace/dotnet_corefx/master/centos7.1_debug_prtest/src/System.Private.Xml/tests/XmlWriter/WriteWithEncodingWithFallback.cs:line 173 ----- Inner Stack Trace ----- at System.Text.EncoderExceptionFallbackBuffer.Fallback(Char charUnknownHigh, Char charUnknownLow, Int32 index) at System.Xml.CharEntityEncoderFallbackBuffer.Fallback(Char charUnknownHigh, Char charUnknownLow, Int32 index) in /mnt/resource/j/workspace/dotnet_corefx/master/centos7.1_debug_prtest/src/System.Private.Xml/src/System/Xml/Core/CharEntityEncoderFallback.cs:line 140 at System.Text.EncoderFallbackBuffer.InternalFallback(Char ch, Char*& chars) at System.Text.ASCIIEncoding.GetBytesFallback(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, EncoderNLS encoder) at System.Text.ASCIIEncoding.GetBytes(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, EncoderNLS encoder) at System.Text.EncoderNLS.Convert(Char[] chars, Int32 charIndex, Int32 charCount, Byte[] bytes, Int32 byteIndex, Int32 byteCount, Boolean flush, Int32& charsUsed, Int32& bytesUsed, Boolean& completed) at System.Xml.XmlEncodedRawTextWriter.<EncodeCharsAsync>d__122.MoveNext() in /mnt/resource/j/workspace/dotnet_corefx/master/centos7.1_debug_prtest/src/System.Private.Xml/src/System/Xml/Core/XmlEncodedRawTextWriterAsync.cs:line 622 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Xml.XmlEncodedRawTextWriter.<FlushBufferAsync>d__121.MoveNext() in /mnt/resource/j/workspace/dotnet_corefx/master/centos7.1_debug_prtest/src/System.Private.Xml/src/System/Xml/Core/XmlEncodedRawTextWriterAsync.cs:line 593 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Xml.XmlEncodedRawTextWriter.<FlushAsync>d__120.MoveNext() in /mnt/resource/j/workspace/dotnet_corefx/master/centos7.1_debug_prtest/src/System.Private.Xml/src/System/Xml/Core/XmlEncodedRawTextWriterAsync.cs:line 534 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Xml.XmlWellFormedWriter.<FlushAsync>d__152.MoveNext() in /mnt/resource/j/workspace/dotnet_corefx/master/centos7.1_debug_prtest/src/System.Private.Xml/src/System/Xml/Core/XmlWellFormedWriterAsync.cs:line 1190

[AndyAyersMS]

We picked up more than just the jit fix with 25019-01. Suspect this latest set of failures might be dotnet/coreclr#9187. @benaadams does this seem possible?

[benaadams]

@AndyAyersMS looks like it could be, it goes through System.Text.ASCIIEncoding.GetBytesFallback

[benaadams]

Making revert PR 😢

[benaadams]

Revert PR dotnet/coreclr#9671

[benaadams]

Will investigate seperately

[danmoseley]

Thanks @benaadams . Perhaps our coverage of the Encoder code is insufficient (as opposed to indirect coverage through XML tests)

[benaadams]

There's a follow up issue https://github.com/dotnet/coreclr/issues/9668 for matching the implementations with corefxlabs which would require a change to how the fallbacks worked; so it looks like the tests in this area would need to be improved before that was attempted.

[AndyAyersMS]

I x-ref'd 9197 initially, not 9187, sorry if that confused anyone. Fixed with edit.

[AndyAyersMS]

Hopefully CoreCLR is back on track now with dotnet/coreclr#9197 reverted via dotnet/coreclr#9671. If I understand the official build schedule, the -02 build should kick off just after midnight PST, so with any luck we will have a successful update around 3AM tomrrow.

[AndyAyersMS]

Seems less likely that this latest failure is a CoreCLR issue.

[AndyAyersMS]

Old failing log here, let's see if it repros.

@dotnet-bot test Innerloop OSX Release Build and Test

[stephentoub]

That failure looks like:
https://github.com/dotnet/corefx/issues/15383
cc: @saurabh500

[stephentoub]

Thanks, all. I'm assuming this is good now.