X509Certificate2/OpenSSL: Unable to open PKCS#12 files with no password and no MAC

[qmfrederik]

If you have a PKCS#12 file which is not protected with a password, and which does not have a MAC entry, opening the file will work on Windows but fails on Linux and Mac (which use OpenSSL).

The following program reproduces the behavior:

using System;
using System.IO;
using System.Security.Cryptography.X509Certificates;

namespace ConsoleApplication
{
    public class Program
    {
        public static void Main(string[] args)
        {
            byte[] rawData = Convert.FromBase64String(
                "MIACAQMwgAYJKoZIhvcNAQcBoIAkgASCA+gwgDCABgkqhkiG9w0BBwGggCSABIID6DCCBRUwggURBgsqhkiG9w0BDAoBAaCCBMAwggS8AgEAMA0GCSqGSIb3DQEBAQUABIIEpjCCBKICAQACggEBAJ9Iw1KuIXuQBnQA3GvlPu2yvXZU8BbM6yBwuypsUdOk4isb8S2+CI/p3Ez8yWMb+HZLQ1QBXDvk2VzZUIJH+xY1oWqu7Hvo9iADkltKbfZ1yjfG5Dy7FatNnVHlGoq7HTK9WlxEVZutrHdpBKHNdHWoXr0xqxS4YtcXnnBZMLlDpVhxUJ6L/X0WRHY52QcLp6ZYfAObU1+l+Ihh7OVP1r8c0HmIH+qL3FaQplhrNjFr6qrCarA42DZjcgShuNOfuvhEJejmgOL7QS0j8h2sT/Fa5oDNwRNJV5vrni8GPbjd4Sbch+90+Oz39tyO8ecvIsKKak67oUbR80EyXNjb0eMCAwEAAQKCAQA8C/AmQSK6NAdav+BYhGl+rj0iWM7RqZqR9i14xrDqOmRQoA4Bknwj1KOKGlnJFQhLf//3sTOWGKWgjQP+uSf8rWcWkq7v31i5pN8NrzdZC/qZoE72XgjDNVUzRE0HM5bERAHGerRTJdu4gEyQuqVGnZxpcknuW7xXHb5K2DS4AiIDfkU9iqkUxa/hEG4HpueBgXv14oRa5z4hPzTbiIcej18f1n3IJA1SsATZ3RjU5B52ni/lYALufdnAJeBycLDU9m7UbqTfXSpb2uWM9yJKCOALHQZzE50/pW3DRQpdfbkoaMucoHqUSsLSbTLkYduHoxb9Bi7A6tgRTQFtOvfhAoGBANC639cEiE4xScrD3s9A/5/8pRoN8ojLEVDk44igGMHz+usHfxxp8YQvmYd5US1P6qPAX1scHKqHDbokd7aDh/5L7TKlwxo+iv7aOPQWJbpEmtsXASsMYfKneifUUxA47GtEYLHQ9Yy3kAratTaHH9K3a6x3wbsk815p77hlRygbAoGB" +
                "AMNbRSmmx/53b1YV6LxOuxzRSyCP5qYswlG2i7H6A+MLmUnxYurFvSptk+vEngmbOJW9rd49bHrK6PUtKBNYLxRWjRUtkgK2Z9PQguc8GD0W7SjHZafpFPETVdKxIVP6a18RB5jNvIuOPA8Z09/Kh80dw9dEPXs1EFQubxEFQ6nZAoGAOZsJgcb/c00JB4vNJzfSFK5eRnWI9RXOHpw864z7qDOUkV7NRuM6Q3f7kDb8H1xJ7o1+A6AbjTieojvESju8wYLk4LB8yvZt1+4T/9FI8kJS1ppfuSi+s4BjJzDjB7weC3CgmxKHYiGbAFPh5T2fm8EBV2Tps6N8AxeLkEFrBIID6ET3AoGAWmveYV7+5rtlXxUY+j/+v2HoBIIBMUIUGRAFW5PyyEoCjNYEQllFTyGXkO0YdwUDppqPq+szNkzNZW6YiKci1Y/Om0vwm7CXvSNgRkJ2GoDpAdcUy4S6dkT3z2eeKXUx41k5aYVBHqENaR23IfljXPwShDTeeA0lWseyUfKE44efRihRAoGAYn+PAVLDWmSKMm6rJDbJne12NYviqOa6n3ZJg0N9I8o0C/xFX1o/IzJF0EPdyVlCj/laIYoUX3O1P5I8YeubcR/JvY/LnCMutgsWlkdbz5c9gfbFVaEXXQPFyKuSaMea6pn+kv9EYtVfgiQ4rZ4aZwe0CWkzfDvIE6AGYgMICMIxPjAXBgkqhkiG9w0BCRQxCh4IAGMAZQByAHQwIwYJKoZIhvcNAQkVMRYEFO3z0SLPYjzwz8nNImJh6EFag+YwAAAAAAAAMIAGCSqGSIb3DQEHAaCAJIAEggPoMIIGETCCBg0GCyqGSIb3DQEMCgEDoIIFvDCCBbgGCiqGSIb3DQEJFgGgggWoBIIFpDCCBaAwggSIoAMCAQICCBulrjADvm1lMA0GCSqGSIb3DQEBBQUAMIGWMQswCQYDVQQGEwJVUzETMBEGA1UECgwKQXBwbGUgSW5jLjEsMCoGA1UECwwjQXBwbGUgV29ybGR3a" +
                "WRlIERldmVsb3BlciBSZWxhdGlvbnMxRDBCBgNVBAMMO0FwcGxlIFdvcmxkd2lkZSBEZXZlbG9wZXIgUmVsYXRpb25zIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTE1MDExMjEyMTIxMFoXDTE2MDExMjEyMTIxMFowgZMxGjAYBgoJkiaJk/IsZAEBDApSSlhWVEU4NjUzMTgwNgYDVQQDDC9pUGhvbmUgRGV2ZWxvcGVyOiBGcmVkZXJpayBDYXJsaWVyICg4VDlVS1VCR1k5KTETMBEGA1UECwwKVENESzVFTEFINzEZMBcGA1UECgwQRnJlZGVyaWsgQ2FybGllcjELMAkGA1UEBhMCVVMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCfSMNSriF7kAZ0ANxr5T7tsr12VPAWzOsgcLsqbFHTpOIrG/EtvgiP6dxM/MljG/h2S0NUAVw75Nlc2VCCR/sWNaFqrux76PYgA5JbSm32dco3xuQ8uxWrTZ1R5RqKux0yvVpcRFWbrax3aQShzXR1qF69MasUuGLXF55wWTC5Q6VYcVCei/19FkR2OdkHC6emWHwDm1NfpfiIYezlT9a/HNB5iB/qi9xWkKZYazYxa+qqwmqwONg2Y3IEggOgBKG405+6+EQl6OaA4vtBLSPyHaxP8VrmgM3BE0lXm+ueLwY9uN3hJtyH73T47Pf23I7x5y8iwopqTruhRtHzQTJc2NvR4wIDAQABo4IB8TCCAe0wHQYDVR0OBBYEFO3z0SLPYjzwz8nNImJh6EFag+YwMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUiCcXCam2GGCL7Ou69kdZxVJUo7cwggEPBgNVHSAEggEGMIIBAjCB/wYJKoZIhvdjZAUBMIHxMIHDBggrBgEFBQcCAjCBtgyBs1JlbGlhbmNlIG9uIHRoaXMgY2VydGlmaWNhdGUgYnkgYW55IHBhcnR5IGFzc3VtZXMgYWNjZXB0YW" +
                "5jZSBvZiB0aGUgdGhlbiBhcHBsaWNhYmxlIHN0YW5kYXJkIHRlcm1zIGFuZCBjb25kaXRpb25zIG9mIHVzZSwgY2VydGlmaWNhdGUgcG9saWN5IGFuZCAEggItY2VydGlmaWNhdGlvbiBwcmFjdGljZSBzdGF0ZW1lbnRzLjApBggrBgEFBQcCARYdaHR0cDovL3d3dy5hcHBsZS5jb20vYXBwbGVjYS8wTQYDVR0fBEYwRDBCoECgPoY8aHR0cDovL2RldmVsb3Blci5hcHBsZS5jb20vY2VydGlmaWNhdGlvbmF1dGhvcml0eS93d2RyY2EuY3JsMA4GA1UdDwEB/wQEAwIHgDAWBgNVHSUBAf8EDDAKBggrBgEFBQcDAzATBgoqhkiG92NkBgECAQH/BAIFADANBgkqhkiG9w0BAQUFAAOCAQEAaIuMydHST4l1fMsaXr51Ejqa00fB3PNY9Rw6oG1cWkSS6RgxAz7AJ7dJCO0tZdqPCX6VKnTHhgMlQrI8tIfU2WcG+sV3tvnAysqRtDPqhHWiR4judlp0ETzoLHJFDbbnEph3NpOYVfUwyCthYL/xv3cF9ohcHvfS02O4MsQl1QKhUzLXEOnjUGgPIb7xmGIP54+TNePhEdOi05ijKr1AO4BgCKjeu7tHYvIuyY9HGOfGyuXsoDrP6F+Jj3VRvYCCZuKIvDnocGHsi9AgaxuOSdp5GQOD6OQvXaOPeJNLf8+1Z1S4h/9lS6VubqH+tp9nvgUuq+zbmHv5iqRadMmz4TE+MBcGCSqGSIb3DQEJFDEKHggAYwBlAHIAdDAjBgkqhkiG9w0BCRUxFgQU7fPRIs9iPPDPyc0iYmHoQVqD5jAAAAAAAAAAAAAAAAAAAAAA");

            File.WriteAllBytes(@"rawData.bin", rawData);

            var certificate = new X509Certificate2(
                rawData: rawData,
                password: null,
                keyStorageFlags: X509KeyStorageFlags.Exportable);

            Console.WriteLine(certificate.SerialNumber);
        }
    }
}

The output on Windows:

1BA5AE3003BE6D65

and on Linux:

Unhandled Exception: Interop+Crypto+OpenSslCryptographicException: error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure
   at Internal.Cryptography.Pal.OpenSslPkcs12Reader.Decrypt(String password)
   at Internal.Cryptography.Pal.PkcsFormatReader.TryReadPkcs12(OpenSslPkcs12Reader pfx, String password, Boolean single, ICertificatePal& readPal, List`1& readCerts)
   at Internal.Cryptography.Pal.PkcsFormatReader.TryReadPkcs12(Byte[] rawData, String password, Boolean single, ICertificatePal& readPal, List`1& readCerts)
   at Internal.Cryptography.Pal.CertificatePal.FromBlob(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at ConsoleApplication.Program.Main(String[] args)

You can get the same error message by running OpenSSL directly on the rawData.bin file (don't provide a password):

openssl.exe pkcs12 -info -in rawData.bin
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
Enter Import Password:
Mac verify error: invalid password?
20876:error:2307E06C:PKCS12 routines:PKCS12_verify_mac:mac absent:.\crypto\pkcs12\p12_mutl.c:119:
20876:error:2307E06C:PKCS12 routines:PKCS12_verify_mac:mac absent:.\crypto\pkcs12\p12_mutl.c:119:

However, OpenSSL will correctly inspect the file if you pass the -nomacver option:

openssl.exe pkcs12 -info -in rawData.bin -nomacver
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
Enter Import Password:
PKCS7 Data
Key bag
Bag Attributes
    friendlyName: cert
    localKeyID: ED F3 D1 22 CF 62 3C F0 CF C9 CD 22 62 61 E8 41 5A 83 E6 30
Key Attributes: <No Attributes>
Enter PEM pass phrase:
PKCS7 Data
Certificate bag
Bag Attributes
    friendlyName: cert
(...)
-----END CERTIFICATE-----

The issue seems to be caused by the absence of a MAC entry in the file. Based on RFC7292, I believe that's valid, and BouncyCastle creates PKCS#12 files without MAC entries if no password is provided.

So net, because

• This file was created by Bouncy Castle, which emits the MAC entry if it is not password protected
• That seems to be compliant with the RFC
• The file loads correctly on Windows
• The file loads correctly using OpenSSL if you specify the -nomacver flag

I would expect .NET Core on Linux & Mac to also be able to read this file.

[bartonjs]

OpenSSL and Windows both use the null password as the empty password when creating PFX files. Does bouncy castle generate a usable file if you specify "" instead of null?

[qmfrederik]

@bartonjs If I use string.Empty as a password when saving the file using Bouncy Castle, and specify string.Empty when opening the file using X509Certificate2 on Linux, everything works because Bouncy Castle creates a MAC entry.

So that's a workaround for me now.

I'd still argue the behavior where X509Certificate2 requires a MAC entry is incorrect (at least, different from X509Certificate2 on Windows), but it's good to have a workaround :)

[vaindil]

Not sure if this is related or if I should open a new issue, but I'm having a similar problem. I'm using certs with a password. They work on Windows but fail on Linux. The specific exception message is this:

error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure

I created the certificates using OpenSSL on Windows Bash, so I don't know if a MAC entry was included. I can't find any results when trying to search for this subject, all I can get are results about Mac OS. I don't know how to generate a cert that will work on Linux as well as Windows.

I used this command to create the cert:

openssl req -x509 -newkey rsa:4096 -keyout mykey.pem -out certificate.pem -days 3650

Then this to convert to PFX:

openssl pkcs12 -export -out mycert.pfx -inkey mykey.pem -in certificate.pem

[bartonjs]

@vaindil Yeah, it seems like a separate issue. But, in order to identify if we'd be able to do anything about it you should send that PFX to a Linux machine with the openssl utility and try

openssl pkcs12 -in mycert.pfx -nokeys

If that can't read it then it's an OpenSSL library problem, and we won't be able to, either. If it can, then there's a fallback path we're missing.

If the command line utility test passes, it'd be great if you could give us a sample PFX that fails.

[vaindil]

@bartonjs Just tested, the openssl command does work correctly. I created a new test certificate, all three parts are attached here.

Key passphrase (testkey.pem): 123456
PFX password (badfinal.pfx): 789789

Generate the key and certificate:

openssl req -x509 -newkey rsa:4096 -keyout testkey.pem -out testcert.pem

Convert to PFX:

openssl pkcs12 -export -out badfinal.pfx -inkey testkey.pem -in testcert.pem

[karelz]

We need a test and likely add one more fallback call.

[sphiecoh]

Any update on this issue , hitting the same error with pfx that is password protected on Ubuntu 16.04.

[bartonjs]

@sphiecoh If your PFX has a password that'd be a different problem, since this is specifically about a PFX with a null password (which is different than "the empty password"). Please open a new issue if you're having a different problem.

[sphiecoh]

nvm I found my issue , password was cleared out of env.

[Condorello]

the mac verify fail also with my Comodo SSL cert, which works fine with other a service. There is something to pass online web server because with "bought" SSL certificate there's not possibility to change the cert-key generation string..

[jeremyVignelles]

Hi, any update on this?

[bartonjs]

@jeremyVignelles Nope, sorry. I'm hoping to get the time to rewrite the PFX loader entirely to work around platform-specific quirks like this. The new Pkcs12Info type in .NET Core 3.0 should be capable of exploring (and extracting data from) a PFX in this state, so you might be able to write a loader for your particular needs.

https://apisof.net/catalog/System.Security.Cryptography.Pkcs.Pkcs12Info

[ghost]

I'm seeing the same issue when creating a PFX file from a key and cert with an empty password (enter twice when prompted) or by passing -password pass: when using openssl. If I pass -nomac to openssl I get a format error when trying to load the PFX.

[inpicksys]

This is annoying, & blocks all my work. Do you have any workaround for this? While .NET Core 3.0 not released yet?

[jeremyVignelles]

The workaround would be to set a password I guess...

EDIT : you can also try with BouncyCastle (the .net core-compatible version)

[ghost]

@inpicksys What I did is just used a password "password" and stored the credential in the source code with a comment explaining that it's a workaround for a bug and included a link to this thread. It's my intent to use no password protection at all for the certificate and I wanted to capture that intent in there as to not raise any alarms when someone eventually notices it.

[stephentoub]

@bartonjs, there's still mention of this issue in the code:

runtime/src/libraries/System.Security.Cryptography.Pkcs/tests/Pkcs12/Pkcs12BuilderTests.cs

Line 406 in 04f2226

// [ActiveIssue("https://github.com/dotnet/corefx/issues/11046", TestPlatforms.AnyUnix)]

runtime/src/libraries/System.Security.Cryptography.Pkcs/tests/Pkcs12/Pkcs12BuilderTests.cs

Line 461 in 04f2226

// [ActiveIssue("https://github.com/dotnet/corefx/issues/11046", TestPlatforms.AnyUnix)]