InvalidCertificate when using examples/client.rs

[odegnome]

I have created a self signed certificate using openssl for the purpose of integrating TLS into a personal project. I am using the examples/server.rs and examples/client.rs files to understand how to handle the TLS connections but for some reason, I'm unable to get the example working with the provided certificate and even the server name. What am I doing wrong here?

Using example/client to interact with the server

The following error shows up whenever I run the client example. I have the run the following command with & without --domain.

$ cargo run --example tokio-rustls-client -- localhost --port 9000 --cafile ../../../auth/demo-CA-certificate.arm --domain Brief
Error: Custom { kind: InvalidData, error: InvalidCertificate(NotValidForName) }

This output shows up on the server

Custom { kind: InvalidData, error: AlertReceived(BadCertificate) }

Expected:

Echo: <ip> - <bytes>

Using openssl s_client

NOTE: The following is server output and not openssl output.

$ openssl s_client -showcerts -connect localhost:9000
Echo: [::1]:52392 - 0

[cpu]

Hi @odegnome

Error: Custom { kind: InvalidData, error: InvalidCertificate(NotValidForName) }

That looks like the relevant error.

Can you share what subject alternate names (SANs) are in the server certificate you generated? Something like: openssl x509 -in $PATH_TO_SERVER_CERT.pem -noout -text | grep -A1 "Subject Alternative Name"

You need to make sure there is a localhost entry if you want to use it to serve HTTPS over https://localhost:9000 with the demo server.

Using openssl s_client

Be wary using s_client to test server behaviour. In my experience it has bad defaults. In this case I suspect you've elided output that shows a verify error.

[odegnome]

Hey @cpu,

Yeah, there is no entry for localhost in SANs as there was no output when I ran your cmd. I thought there might be an issue with my usage of the client.rs example. As, s_client did not give the InvalidCertificate error. However, here is a shortened output of s_client, which does show verify error. I am still confused why one gives InvalidCertificate but not the other.

Connecting to ::1                                    
CONNECTED(00000005)                                                                                                            
Can't use SSL_get_servername                                                                                                   
depth=0 C=IN, ST=Uttar-Pradesh, L=Noida, O=Brief Pvt Ltd, CN=Brief                                                             
verify error:num=18:self-signed certificate                                                                                    
verify return:1                            
depth=0 C=IN, ST=Uttar-Pradesh, L=Noida, O=Brief Pvt Ltd, CN=Brief                                                             
verify return:1                                                                                                                
---                              
Certificate chain                
 0 s:C=IN, ST=Uttar-Pradesh, L=Noida, O=Brief Pvt Ltd, CN=Brief                                                                
   i:C=IN, ST=Uttar-Pradesh, L=Noida, O=Brief Pvt Ltd, CN=Brief                                                                
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256                                                                       
   v:NotBefore: Dec 27 12:51:46 2024 GMT; NotAfter: Dec 18 12:51:46 2025 GMT                                                   
---                                                                                                                            
<redacted to keep msg short>

Nevertheless, I'll add the localhost SAN and try again.

[cpu]

As, s_client did not give the InvalidCertificate error. However, here is a shortened output of s_client, which does show verify error. I am still confused why one gives InvalidCertificate but not the other.

As mentioned openssl s_client has really bad defaults. You need to add -verify_hostname localhost in order to have the (secure) default behaviour that the Rustls example client gets for free.

If you add -verify_hostname localhost you should see an error like: "Verify return code: 62 (hostname mismatch)"

[odegnome]

If you add -verify_hostname localhost you should see an error like: "Verify return code: 62 (hostname mismatch)"

Yes, it does. However, the server does not show any error. I also tried with -verify_hostname Brief which did not result in this error(CN=Brief in cert). Is this expected?

I really thought I was starting to understand TLS, but back to square one. Thank you for cautioning against s_client.

[ctz]

The main issue here is that modern TLS does not look at the certificate subject commonName for DNS names. There's quite a legacy of "make a certificate using OpenSSL command line" blog posts that draw you to make certificates that don't work outside of OpenSSL (eg, it has not worked in golang since 2020, or firefox since 2016, or chrome since 2017, and rustls never supported it).

[odegnome]

The main issue here is that modern TLS does not look at the certificate subject commonName for DNS names. There's quite a legacy of "make a certificate using OpenSSL command line" blog posts that draw you to make certificates that don't work outside of OpenSSL (eg, it has not worked in golang since 2020, or firefox since 2016, or chrome since 2017, and rustls never supported it).

Yeah, It seems I followed one of the legacy posts. Thank you!

[cpu]

Yeah, It seems I followed one of the legacy posts. Thank you!

You might have better luck with a more friendly/modern tool like minica.

I'm going to close this for now since it seems we reached a mutual understanding of the cause of the error and there's no change to make in this repo. Best of luck,

[odegnome]

An update, just in case someone faces the same problem. I followed this post to generate new keys which resolved this issue.