Handling behaviors implemented in platform verifier

[jsha]

In general, root programs maintain both a trust store and a platform verifier, and the behavior of the two is linked. Distrusts may be implemented first in the verifier (subject to various conditions), and only much later by removal from the trust store.

Using #24, I built a list of trust anchor subjects on three platforms (each of the below is sorted):

trust anchors on macOS 11.2.3

trust anchors on my Windows 10 instance

trust anchors on Ubuntu 20.04

One problem I notice on macOS: there are two StartCom trust anchors in the list, but:

Apple products will block certificates from WoSign and StartCom root CAs if the "Not Before" date is on or after 1 Dec 2016 00:00:00 GMT/UTC

Some background on StartCom here.

This is an example of a fairly common strategy that root programs follow when distrusting a CA. Rather than immediately remove a trust anchor, which would break websites and impact many users, a root program will implement distrust in their platform verifier based on certain criteria. In this case, it's the notAfter date.

As a related example, in 2015 Chrome (while not a root program) implemented a CT requirement for CNNIC and for Symantec, ahead of the general CT requirement for all certificates.

Root programs may also block certain intermediates that don't show up in the trust store. For instance, in the Apple example linked above, WoSign's certificates were never in the trust store directly, but were trusted due to a cross-sign; Apple blocked that cross-sign.

We probably won't be able to come up with a comprehensive list of additional behaviors implemented by platform verifiers, since they are not in general guaranteed to be documented, but we should look for examples like the above and try to implement them here.

Mozilla has some guidance for bundlers of their trust store:

The decisions Mozilla makes with regards to the inclusion or exclusion of CA certificates in its root store are directly tied to the capabilities and behaviours of the software Mozilla distributes. Sometimes, a security change is made wholly or partly in the software instead of the root store. Further, Mozilla does not promise to take into account the needs of other users of its root store when making such decisions.

Therefore, anyone considering bundling Mozilla's root store with other software needs to be aware of the issues surrounding providing a root store, and committed to making sure that they maintain security for their users by carefully observing Mozilla's actions and taking appropriate steps of their own. On a best-efforts basis, Mozilla maintains a list of the additional things users of our store might need to consider.

[ctz]

In retrospect this crate provides the wrong abstraction to solve these problems -- getting some root certificates to use with webpki is one thing, but if we really want to provide platform-specific certificate validation (including the rules mentioned above, but most importantly tracking changes to them accurately over time) we need to hook into the platform's certificate verification apparatus.

[ctz]

So, to clarify, I think the extent that we fix this here should be limited to a denylist of known-bad root certificates that have special handling in the platform verifier. But this is ultimately unsatisfying, and we probably want a different crate that provides implementations of rustls::ServerCertVerifier (to start) backed by the windows and mac verifiers.

[jsha]

Related issue for Go: golang/go#46287

[jsha]

we probably want a different crate that provides implementations of rustls::ServerCertVerifier (to start) backed by the windows and mac verifiers.

Is there a particular reason to do this as a different crate? It seems to me the best approach would be:

• Add an implementation of ServerCertVerifier to this crate.
• Mark the existing "get a list of roots" methods deprecated.

That way, there's a clear way to let people know they should change their code. When they update rustls-native-certs they'll get a deprecation warning and can switch to the ServerCertVerifier path.

If we do this as an entirely separate crate, I suspect a lot of code will never switch to the newer, more correct crate.

[djc]

Given that ServerCertVerifier is guarded with dangerous_configuration on the rustls side, there is some conflict between exposing a public implementation of the trait in a different crate while maintaining the boundary that dangerous_configuration puts up for users shooting themselves in the foot. (I suppose one solution might be to have the platform-native verifier implementation live in rustls itself, potentially enabled through a different feature flag.)

[jsha]

Given that ServerCertVerifier is guarded with dangerous_configuration on the rustls side, there is some conflict between exposing a public implementation of the trait in a different crate while maintaining the boundary that dangerous_configuration puts up for users shooting themselves in the foot.

It should be possible to make the ServerCertVerifier trait always-public, while still feature-guarding ConfigBuilder::with_custom_certificate_verifier.

[djc]

And then you have no way to use a ServerCertVerifier that's defined externally to the rustls crate, so doesn't help?

[jsha]

Aha, I was imagining that it would be up to the user to enable the dangerous_configuration flag. But we don't want users to have to enable that flag just to use rustls-native-certs, which we don't consider dangerous.

Perhaps dangerous_configuration is not really the right tool for the job. If our goal is to discourage people from disabling certificate validation, we do better by providing alternatives (like SSL_CERT_FILE) for the situations when people commonly disable it. And the folks who really need to turn it off are going to wind up doing so anyhow, even if there are big warnings in the way.

[njsmith]

Another complication: the platform verifier interfaces internally do blocking I/O – they do stuff like phone home to MS/Apple to get the latest updates to the trust store. So IIUC rustls's handshake interfaces are themselves somewhat misdesigned: if you're doing async network I/O, then you want to run most of the handshake in async mode, but then when it's time to verify certs you need to push that off into a thread pool.

So to use the platform verifier, rustls would need to expose some way to tell the application "okay, pause the handshake here, run this thunk in a thread, and then when it's done give me the result and go back to driving the handshake normally".

[njsmith]

References:

• My notes that I was looking at: Discussion: SSL backend python-trio/trio#1145 (comment)
• Dart/Flutter hitting multi-second freezes because they were using the platform verifier from the main thread: SecureSocket handshake can stall the main thread for long time dart-lang/sdk#41519
• BoringSSL extended the classic WANT_READ/WANT_WRITE to add a third state, WANT_CERTIFICATE_VERIFY; more details.

[njsmith]

A friend who's an expert on SwiftNIO happened to see this bug and said:

SwiftNIO SSL does exactly this on the Apple platforms, including thunking out to a new thread.

...and also confirmed that this is required for using Security.framework's trust evaluation functions.

Their current code using BoringSSL: https://github.com/apple/swift-nio-ssl/blob/main/Sources/NIOSSL/SecurityFrameworkCertificateVerification.swift

[Ralith]

So to use the platform verifier, rustls would need to expose some way to tell the application "okay, pause the handshake here, run this thunk in a thread, and then when it's done give me the result and go back to driving the handshake normally".

rustls/rustls#787 recently addressed a class of issues like this on the server side; I think the same pattern would apply here as well.

[jsha]

For the macOS trust store, there is now a published page listing certificates with "Additional Certificate Configuration Data"; essentially certificates that are blocked: https://support.apple.com/en-us/HT212865.

[complexspaces]

FWIW, to close a little bit of the loop here, https://github.com/rustls/rustls-platform-verifier now exists as an alternative/replacement for rustls-native-certs to ensure that these behaviors are accounted for. It verifies certificates using the platform verifier interfaces. Notably though it doesn't yet do anything special regarding I/O today.

[cpu]

I think we've converged on supporting platform verifier behaviour from the crate that complexspaces linked above. I'm going to close this issue but if there's still interest in discussing whether the rustls-native-certs crate should be changed in some way we can certainly re-open. Thanks!