X509Chain is not consistent with NotValidForUsage between Windows and Linux

[bartonjs]

This is tracking a test that will be committed disabled, more variations are probably worth pursuing.

• Build a self-signed certificate, no EKU
• Build an intermediate CA, no EKU.
• Build an end-entity certificate good for TLS Client
• Feed all of that into an appropriately-trusting X509Chain, test for the TLS Server EKU in ApplicationPolicy

Windows reports NotValidForUsage only at the end-entity certificate level in the chain. Linux reports it for all three certificates.

This particular test result was from a test that did the above as well as revoked the intermediate; it probably also applies with no revocation checking (or a non-revoked chain).

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, Revoked, NotValidForUsage | RevocationStatusUnknown | OfflineRevocation]
Actual:   X509ChainStatusFlags[] [NotValidForUsage, Revoked | NotValidForUsage, NotValidForUsage | RevocationStatusUnknown | OfflineRevocation]

[v-haren]

failed again in job: runtime-libraries outerloop 20200423.2

failed test: System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeRootAndEndEntity(pkiOptions: IssuerRevocationViaOcsp | EndEntityRevocationViaCrl | RootAuthorityHasDesignatedOcspResponder)

Error message

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, NoError, Revoked]
Actual: X509ChainStatusFlags[] [NoError, RevocationStatusUnknown | OfflineRevocation, Revoked]


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1263
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertRevocationLevel(X509Chain chain, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1178
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleRevocationBody(ChainHolder holder, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked, Boolean testWithRootRevocation) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1119
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeRootAndEndEntity>b__12_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 236
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1234
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeRootAndEndEntity(PkiOptions pkiOptions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 217

[v-haren]

failed again in job: runtime-libraries outerloop 20200728.3

failed test: System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEndEntityWithExpiredRevocation(pkiOptions: IssuerRevocationViaCrl | OcspEverywhere | IssuerAuthorityHasDesignatedOcspResponder)

Error message

Assert.Equal() Failure
Expected: X509ChainStatusFlags[] [NoError, NoError, Revoked]
Actual:   X509ChainStatusFlags[] [NoError, NoError, RevocationStatusUnknown | OfflineRevocation]


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertChainStatus(X509Chain chain, X509ChainStatusFlags rootStatus, X509ChainStatusFlags issrStatus, X509ChainStatusFlags leafStatus) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1272
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.AssertRevocationLevel(X509Chain chain, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1187
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleRevocationBody(ChainHolder holder, X509Certificate2 endEntityCert, Boolean rootRevoked, Boolean issrRevoked, Boolean leafRevoked, Boolean testWithRootRevocation) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1128
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.<>c.<RevokeEndEntityWithExpiredRevocation>b__24_0(CertificateAuthority root, CertificateAuthority intermediate, X509Certificate2 endEntity, ChainHolder holder) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 879
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1243
   at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEndEntityWithExpiredRevocation(PkiOptions pkiOptions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 869

[bartonjs]

There are two tests which are showing apparently contradictory data:

• ChainTests.BuildChain_FailOnlyApplicationPolicy
  • This test expects to see NotValidForUsage in all positions
• DynamicRevocationTests.RevokeIntermediate_PolicyErrors_NotTimeValid
  • This test expects to see NotValidUsage only in the leaf/EE position

It turns out, they're not testing the same thing.

In DynamicRevocationTests.RevokeIntermediate_PolicyErrors_NotTimeValid we have a pristine chain, built just for that test. The chain is built with CustomRootTrust, and the root certificate has no EKU extension. So in this test Windows is reporting that the requested usage was not permitted by the EE EKU, but that the issuer and root were both valid for that purpose.

In ChainTests.BuildChain_FailOnlyApplicationPolicy we're using system trust with a root that was installed by/with the OS. The root certificate itself has no EKU, so the test looks isomorphic to DynamicRevocationTests.RevokeIntermediate_PolicyErrors_NotTimeValid. However, the Windows Certificate Store system supports virtual properties. The relevant root to this test (DigiCert Baltimore Root) has an EKU override property, limiting the scope to TLS-Server, TLS-Client, CodeSign, S/MIME, or Timestamp. Since the root is scoped to only those 5 purposes, Windows reports that not only was the leaf/EE not valid for the requested purpose (0.1.2.3.4), but also the issuer (scoped by the root) and the root (scoped by the virtual property) were not valid for the requested purpose. Using custom root trust with a clean copy of the root cert shows the same { NotValidForUsage, OK, OK } as the dynamic test (it might actually be NotValid/NotValid/OK, the intermediate looks like it might be scoped either by the CA or with a virtual property). Similarly, checking for the codesign EKU (1.3.6.1.5.5.7.3.3) instead of the made up EKU changes the results.

This means that the policy validation step of the chain needs to not be boolean, but report the errors at the level they occur.

Since this is a non-trivial change and the issue is self-reported with no community weigh-in, moving to 7.0. But at least now it's understood.