[ARM64] Multiple test failures : /System.Security.Cryptography.Algorithms/tests/AesCcmTests.cs

[dmklementiev-zz]

Note that the failure occurs on Arm64 only!

The test: https://github.com/dmklementiev/corefx/blob/master/src/System.Security.Cryptography.Algorithms/tests/AesCcmTests.cs has failed 37 times (details here https://mc.dot.net/#/user/dmklementiev/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Fmaster~2F/test~2Ffunctional~2Fcli~2F/f8e7a02f30f1062cd70104bbf1a364e8acedb3bd/workItem/System.Security.Cryptography.Algorithms.Tests) with the following details:

Unhandled Exception of Type Xunit.Sdk.EqualException
Message :
Assert.Equal() Failure
Expected: Byte[] [0, 0, 0, 0, 0, ...]
Actual: Byte[] [163, 189, 216, 244, 17, ...]
Stack Trace :
at System.Security.Cryptography.Algorithms.Tests.AesCcmTests.AesCcmNistTestsTamperCiphertext(AEADTest testCase) in /mnt/j/workspace/dotnet_corefx/master/linux-TGroup_netcoreapp+CGroup_Release+AGroup_arm64+TestOuter_false_prtest/src/System.Security.Cryptography.Algorithms/tests/AesCcmTests.cs:line 381

Similar failures happened for the same StackTrace for different Actual: Byte[] values. Just a few for example:
Actual: Byte[] [43, 61, 80, 100, 121, ...]
Actual: Byte[] [221, 239, 2, 22, 43, ...]
Actual: Byte[] [213, 231, 250, 14, 35, ...]

The same at Stack Trace:

at System.Security.Cryptography.Algorithms.Tests.AesCcmTests.AesCcmNistTestsTamperTag(AEADTest testCase) in /mnt/j/workspace/dotnet_corefx/master/linux-TGroup_netcoreapp+CGroup_Release+AGroup_arm64+TestOuter_false_prtest/src/System.Security.Cryptography.Algorithms/tests/AesCcmTests.cs:line 359

The same at Stack Trace:

at System.Security.Cryptography.Algorithms.Tests.AesCcmTests.InplaceEncryptTamperTagDecrypt() in /mnt/j/workspace/dotnet_corefx/master/linux-TGroup_netcoreapp+CGroup_Release+AGroup_arm64+TestOuter_false_prtest/src/System.Security.Cryptography.Algorithms/tests/AesCcmTests.cs:line 319

[vcsjones]

cc @krwq

[dmklementiev-zz]

I am going to disable the corresponding tests this morning before this is enabled in regular CI. Created this issues to add [ActiveIssue(32710, TestPlatforms.AnyUnix)] to the corresponding tests before committing.

[krwq]

@dmklementiev do we have any way to disable these tests on ARM only?

[dmklementiev-zz]

I would love to, but it doesn't seem you have this kind of granularity - look at how TestPlatforms are defined:

https://github.com/dotnet/buildtools/blob/2f0dec8eba53942a758910ea7b4cc79e4a7d2c36/src/xunit.netcore.extensions/TestPlatforms.cs#L10-L19

I was also told by Wes that picking Arm64 only is not possible today. Please let me know if I am missing something, so I will gladly change it to be more granular - for Arm64 only.

[weshaggard]

test disabled with dotnet/corefx#32566

@dmklementiev do we have any way to disable these tests on ARM only?

Currently there isn't a way but there is a PR someone started but never got finished dotnet/buildtools#1659