Mutual authentication with tls

[liqiangno1]

Describe your question

I have a mqtt broker like emqx，it support mutual authentication with tls, I connect the broker successfully by using Mqtt.fx client software. but I don't known how to set the tls option with MQTTnet, The wiki is also not clearly. Can you can help me?

The Mqtt.fx screenshot.

Which project is your question related to?

• Client
  Mqttnet version: 3.0.5
  Net core version: 2.2

[liqiangno1]

#115 #695
I saw these issues , but still have no idea.

[liqiangno1]

.WithTls(o =>
    {
        o.UseTls = true;
        o.Certificates = new List<byte[]>
        {
            new X509Certificate(@"C:/Users/liqia/Downloads/server/ca.pem", "").Export(X509ContentType.Cert),
            new X509Certificate(@"C:\Users\liqia\Downloads\newclient\client.pfx", "").Export(X509ContentType.Cert)
        };
        o.CertificateValidationCallback = (x509Certificate, chain, sslPolicyErrors, mqttClientTcpOptions) => true;
    })

I have generated the .pfx cert file and imported them. And I received the following exception:

The message received was abnormal or incorrectly formatted

[SeppPenner]

You might take a look at this project: https://github.com/SeppPenner/NetCoreMQTTExampleJsonConfig/blob/b5159009f33e23b425313ac8cfcb361474c0688b/NetCoreMQTTExampleJsonConfig/Program.cs#L47. I'm using a certificate succssfully in there.

[robin-jones]

try .Export(X509ContentType.SerializedCert)

There is also a way to use the certs in their original pem and key format, but I am still working on a blog post for that!

[liqiangno1]

You might take a look at this project: https://github.com/SeppPenner/NetCoreMQTTExampleJsonConfig/blob/b5159009f33e23b425313ac8cfcb361474c0688b/NetCoreMQTTExampleJsonConfig/Program.cs#L47. I'm using a certificate succssfully in there.

it's a server demo. but I need a client demo

[liqiangno1]

try .Export(X509ContentType.SerializedCert)

There is also a way to use the certs in their original pem and key format, but I am still working on a blog post for that!

I run it successfully when I import the cert to windows certmgr. but when run in linux system, it still has a exception.
MQTTnet.Exceptions.MqttCommunicationException: Authentication failed, see inner exception. ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> Interop+OpenSsl+SslException: SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL. ---> Interop+Crypto+OpenSslCryptographicException: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

[SeppPenner]

it's a server demo. but I need a client demo

Sorry, I misunderstood you here.

[SeppPenner]

This is also something to be documented once it works. I can do that if we find a solution.

[networkfusion]

So with my code, I can connect to AWS IoT on windows, but not on Linux. In my case I receive:

I have a suspicion that both of these issues might be related to https://github.com/dotnet/corefx/issues/34740#

my code is as follows:

            var options = new MqttClientOptionsBuilder()
                    .WithClientId(mqttSettings.ClientId)
                    .WithTcpServer(mqttSettings.Endpoint, mqttSettings.Port)
                    .WithKeepAlivePeriod(new TimeSpan(0, 0, 0, 300))
                    .WithTls(new MqttClientOptionsBuilderTlsParameters
                    {
                        UseTls = true,
                        CertificateValidationCallback = (X509Certificate x, X509Chain y, SslPolicyErrors z, IMqttClientOptions o) =>
                        {
                            // TODO: Check conditions of certificate by using above parameters.
                            return true;
                        },
                        AllowUntrustedCertificates = false,
                        IgnoreCertificateChainErrors = false,
                        IgnoreCertificateRevocationErrors = false,
                        Certificates = new List<byte[]>()
                        {
                            new X509Certificate(mqttSettings.RootCaPath).Export(X509ContentType.Cert), //TODO: need to find a way to dispose?
                            new X509Certificate2(
                                    Crypto.GetCertificateFromPEMstring(
                                    File.ReadAllText(mqttSettings.ClientCertificatePath),
                                    File.ReadAllText(mqttSettings.PrivateKeyPath),
                                    ""))
                                .Export(X509ContentType.Pfx) //TODO: need to find a way to dispose?
                        }
                    })
                    .WithProtocolVersion(MqttProtocolVersion.V311)
                    .Build();

with helper method:

        /// <summary>
        /// Creates X509 certificate
        /// </summary>
        /// <param name="publicCertificate">PEM string of public certificate.</param>
        /// <param name="privateKey">PEM string of private key.</param>
        /// <param name="password">Password for certificate.</param>
        /// <returns>An instance of <see cref="X509Certificate2"/> rapresenting the X509 certificate.</returns>
        public static X509Certificate2 GetCertificateFromPEMstring(string publicCertificate, string privateKey, string password)
        {
            X509Certificate2 certificate = new X509Certificate2(GetBytesFromPemString(publicCertificate, PemStringType.Certificate), password);
            var privateKeyBytes = GetBytesFromPemString(privateKey, PemStringType.RsaPrivateKey);
            using var rsa = RSA.Create();
            rsa.ImportRSAPrivateKey(privateKeyBytes, out _);
            X509Certificate2 certificateWithKey = certificate.CopyWithPrivateKey(rsa);
            return certificateWithKey;
        }

        private static byte[] GetBytesFromPemString(string pemString, PemStringType type)
        {
            string header, footer;

            switch (type)
            {
                case PemStringType.Certificate:
                    header = "-----BEGIN CERTIFICATE-----";
                    footer = "-----END CERTIFICATE-----";
                    break;
                case PemStringType.RsaPrivateKey:
                    header = "-----BEGIN RSA PRIVATE KEY-----";
                    footer = "-----END RSA PRIVATE KEY-----";
                    break;
                case PemStringType.PublicKey:
                    header = "-----BEGIN PUBLIC KEY-----";
                    footer = "-----END PUBLIC KEY-----";
                    break;
                default:
                    return null;
            }

            int start = pemString.IndexOf(header) + header.Length;
            int end = pemString.IndexOf(footer, start) - start;
            return Convert.FromBase64String(pemString.Substring(start, end));
        }

        

        private enum PemStringType
        {
            Certificate,
            RsaPrivateKey,
            PublicKey
        }

FYI, This code needs .net core 3 due to rsa.ImportRSAPrivateKey(..)

[EmanueleGiuliano]

Hi everyone, i'm facing a similar problem with my code. I found that on linux it doesn't send the certificate(we checked packets in the network and we can say that the certificate is not sent when we are in linux) to the broker (i'm tring to make it work an aks), while on a windows VM it connects and reads messages.

linux image is: mcr.microsoft.com/dotnet/core/aspnet:3.0-alpine

here is my code:

` var mqttBrokerCertPubKey = new X509Certificate(cAcertificatePath);
var mqttDmiClinetCert = new X509Certificate2(clientPfxCertificatePath, pfxPassword);

        //var mqttClientCert = new X509Certificate(clientCrtCertPath, "", X509KeyStorageFlags.Exportable);
        
        Console.WriteLine($"Configuration: topic-> ${topicSubscription}, clientId-> ${mqClientId}, address->${mqTcpAddress}, caPath->${cAcertificatePath}, certPath->${clientPfxCertificatePath}");
        
        var options = new ManagedMqttClientOptionsBuilder()
        .WithAutoReconnectDelay(TimeSpan.FromSeconds(5))
        .WithClientOptions(new MqttClientOptionsBuilder()
                    .WithProtocolVersion(MQTTnet.Formatter.MqttProtocolVersion.V311)
                    .WithClientId(mqClientId)
                    .WithTcpServer(mqTcpAddress, Configuration.GetValue<int>("port"))
                    .WithCommunicationTimeout(new TimeSpan(0, 2, 30))
                    .WithCleanSession()
                    .WithTls(new MqttClientOptionsBuilderTlsParameters()
                    {
                        UseTls = true,
                        AllowUntrustedCertificates = true,
                        Certificates = new[] { mqttBrokerCertPubKey.Export(X509ContentType.Cert), mqttDmiClinetCert.Export(X509ContentType.Pfx) },
                        IgnoreCertificateChainErrors = true,
                        IgnoreCertificateRevocationErrors = true,                             
                        SslProtocol = SslProtocols.None,
                        CertificateValidationCallback = (X509Certificate x, X509Chain y, SslPolicyErrors z, IMqttClientOptions o) =>
                        {

                            Console.WriteLine("Certificate--> issuer: " + x.Issuer + " subject: " + x.Subject);
                            return true;

                        }                            
                    })
                    .Build()
            )
        .Build();

        mqttClient = new MqttFactory().CreateManagedMqttClient();
        bool status = false;
        await mqttClient.StartAsync(options);

         mqttClient.ConnectingFailedHandler = new ConnectingFailedHandlerDelegate((ManagedProcessFailedEventArgs err)=>{
            Console.WriteLine("Error connecting to broker");
            Console.WriteLine(err.Exception.Message);
        });`

[robin-jones]

not sure if it helps, but I found that @networkfusion 's example works on Linux (using Docker) if I exclude the RootCA and set IgnoreCertificateRevocationErrors = true . Obviously you have to set the private cert and key as "Content" so that the docker image has them (probably a better way). I am guessing that it would be possible to to add the rootCA to the Linux key store so that it works as expected, but I haven't found a way yet...

[albigi]

we were able to understand the cause of the issue reported by @EmanueleGiuliano.
the client certificate we were passing to the MqttClientOptions was not trusted by our Linux containers, nor it was installed in the personal certificate store. Due to such limitations it was not being picked up during the TLS handshake upon receiving the list of supported trusted CAs from the broker endpoint.

bottomline, we solved the problem by adding the certificate to the cert store. we achieved this by running the lines below before configuring the MqttClient:

using (var certStore = new X509Store(StoreName.My, StoreLocation.CurrentUser)) { certStore.Open(OpenFlags.ReadWrite); var cert = new X509Certificate2(); //read the certificate from somewhere here certStore.Add(cert); }

this solved out issues both on Linux and Windows.

At this stage I am wondering if installing the certificate in the personal store would make passing the certificate list to the MqttClientOptions useless (SChannel in Windows and OpenSSL or other PKI stacks in Linux should be able to figure out the right cert to use once it's available in the cert store). Perhaps this step should be included into the library code itself?

also, it would be interesting to test if this same behavior could be reprod on a previous version of .NET Core

[zhaopeiym]

It cannot solve the problem, under docker aspnet:3.0-alpine

Windows is fine, but Linux reports errors. SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.

[zhaopeiym]

在docker aspnet：3.0-alpine下无法解决问题

Windows很好，但是Linux报告错误。 SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.

已解决，两种方案实现TLS连接：

1、通过WebSocket 
var mqttClientOptions = new MqttClientOptionsBuilder()
             .WithClientId(clientID)
             .WithWebSocketServer("broker.hivemq.com:443/mqtt")
             .WithCredentials(userName, password)
             .WithTls();

2、只需要导入client.pfx，不需要ca.crt
var clientCert = new X509Certificate2(AppConfig.MqttPfxFile); 
var mqttClient = new MqttFactory().CreateManagedMqttClient();
var mqttClientOptions = new MqttClientOptionsBuilder()
             .WithClientId(clientID)
             .WithTcpServer(address, port)
             .WithCredentials(userName, password)
             .WithTls(new MqttClientOptionsBuilderTlsParameters()
             {
                 UseTls = true,
                 SslProtocol = System.Security.Authentication.SslProtocols.Tls12,
                 CertificateValidationHandler = (o) =>
                 {
                     return true;
                 },
                 Certificates = new []{                                      
                     clientCert,
                 }
             });

生成pfx：openssl pkcs12 -export -in client.crt -inkey client.key -out client.pfx

[janhumble]

I'm trying to replicate "self signed certificates in keystores" in MQTT.fx. What would the TLS configuration look like to replicate the configuration below, ideally fully programmatically?

[MrzJkl]

I still get this issue in .NET 6...

Adding CA Certificate and Client certificate to the Certstore does not fix it.

Anyone else having this Problem?

[PWNTechIT]

在docker aspnet：3.0-alpine下无法解决问题
Windows很好，但是Linux报告错误。 SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.

已解决，两种方案实现TLS连接：

1、通过WebSocket 
var mqttClientOptions = new MqttClientOptionsBuilder()
             .WithClientId(clientID)
             .WithWebSocketServer("broker.hivemq.com:443/mqtt")
             .WithCredentials(userName, password)
             .WithTls();

2、只需要导入client.pfx，不需要ca.crt
var clientCert = new X509Certificate2(AppConfig.MqttPfxFile); 
var mqttClient = new MqttFactory().CreateManagedMqttClient();
var mqttClientOptions = new MqttClientOptionsBuilder()
             .WithClientId(clientID)
             .WithTcpServer(address, port)
             .WithCredentials(userName, password)
             .WithTls(new MqttClientOptionsBuilderTlsParameters()
             {
                 UseTls = true,
                 SslProtocol = System.Security.Authentication.SslProtocols.Tls12,
                 CertificateValidationHandler = (o) =>
                 {
                     return true;
                 },
                 Certificates = new []{                                      
                     clientCert,
                 }
             });

生成pfx：openssl pkcs12 -export -in client.crt -inkey client.key -out client.pfx

Your solution works fine! Thanks for saving me another 2 hours of headaches!