unexpected test_rsa failure

[jku]

Failure on CI (python 3.10 Ubuntu): openssl error message is different from expected

======================================================================
FAIL: test_rsa (tests.test_interface.TestInterfaceFunctions)
Test RSA key _generation and import interface functions.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/securesystemslib/securesystemslib/tests/test_interface.py", line 315, in test_rsa
    self.assertTrue(
AssertionError: False is not true : expected: 'Bad decrypt. Incorrect password?' got: 'RSA (public, private) tuple cannot be generated from the encrypted PEM string: ('Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', [_OpenSSLErrorWithText(code=503841036, lib=60, reason=524556, reason_text=b'error:1E08010C:DECODER routines::unsupported'), _OpenSSLErrorWithText(code=109052027, lib=13, reason=123, reason_text=b'error:0680007B:asn1 encoding routines::header too long'), _OpenSSLErrorWithText(code=109052006, lib=13, reason=102, reason_text=b'error:06800066:asn1 encoding routines::bad object header'), _OpenSSLErrorWithText(code=109576458, lib=13, reason=524554, reason_text=b'error:0688010A:asn1 encoding routines::nested asn1 error'), _OpenSSLErrorWithText(code=109052027, lib=13, reason=123, reason_text=b'error:0680007B:asn1 encoding routines::header too long'), _OpenSSLErrorWithText(code=109052006, lib=13, reason=102, reason_text=b'error:06800066:asn1 encoding routines::bad object header'), _OpenSSLErrorWithText(code=109576458, lib=13, reason=524554, reason_text=b'error:0688010A:asn1 encoding routines::nested asn1 error')])' (row 4)

The openssl output with some linebreaks:

RSA (public, private) tuple cannot be generated from the encrypted PEM string:
(
  'Could not deserialize key data. The data may be in an incorrect format, it may be encrypted with an unsupported algorithm, or it may be an unsupported key type (e.g. EC curves with explicit parameters).', 
  [
    _OpenSSLErrorWithText(code=503841036, lib=60, reason=524556, reason_text=b'error:1E08010C:DECODER routines::unsupported'),
    _OpenSSLErrorWithText(code=109052027, lib=13, reason=123, reason_text=b'error:0680007B:asn1 encoding routines::header too long'),
    _OpenSSLErrorWithText(code=109052006, lib=13, reason=102, reason_text=b'error:06800066:asn1 encoding routines::bad object header'),
    _OpenSSLErrorWithText(code=109576458, lib=13, reason=524554, reason_text=b'error:0688010A:asn1 encoding routines::nested asn1 error'),
    _OpenSSLErrorWithText(code=109052027, lib=13, reason=123, reason_text=b'error:0680007B:asn1 encoding routines::header too long'),
    _OpenSSLErrorWithText(code=109052006, lib=13, reason=102, reason_text=b'error:06800066:asn1 encoding routines::bad object header'),
    _OpenSSLErrorWithText(code=109576458, lib=13, reason=524554, reason_text=b'error:0688010A:asn1 encoding routines::nested asn1 error')
  ]
)

Failing run: https://github.com/secure-systems-lab/securesystemslib/actions/runs/3546570396/jobs/5955750691
Earlier successful run with only unrelated code changes https://github.com/secure-systems-lab/securesystemslib/actions/runs/3530872651/jobs/5923516701

• same OS image
• same Python version
• same cryptography version

Issue seems to be an unexpected message from openssl... but I don't see how openssl could have changed given the above...

[jku]

happened again, different python version but same test
https://github.com/secure-systems-lab/securesystemslib/actions/runs/3910375944/jobs/6682507251

[lukpueh]

same test, this time on windows
https://github.com/secure-systems-lab/securesystemslib/actions/runs/4467596645/jobs/7847262459?pr=542

[lukpueh]

The only thing that changes between test runs is the private key:

securesystemslib/tests/test_interface.py

Line 159 in 515c56d

_generate_and_write_rsa_keypair(filepath=fn_encrypted, password=pw)

Would be interesting to see what the private key looks like, when the test fails.

[lukpueh]

I was able to trigger the error after a couple (100 or so) iterations on a Debian box using this script:

from securesystemslib import keys

path = "key"

while True:
    print("Create key")
    key_pair = keys.generate_rsa_key(3072)
    encrypted_key = keys.create_rsa_encrypted_pem(key_pair["keyval"]["private"], "pw")

    try:
        keys.import_rsakey_from_private_pem(encrypted_key, "rsassa-pss-sha256", "bad pw")

    except Exception as e:
        if "Could not deserialize key" in str(e):
           breakpoint()

... but there wasn't anything unusual about that key.

[jku]

So likely this points to some flakiness in openssl 😬

Should we make the test here succeed on either error message?

[lukpueh]

Oh interesting, it depends on the password. I think I'll leave this for the crypto experts to debug. Maybe worth creating in issue on pyca/cryptography...

@jku, can you reproduce this?

from cryptography.hazmat.primitives.serialization import load_pem_private_key

encrypted_rsa_key = b"-----BEGIN RSA PRIVATE KEY-----\nProc-Type: 4,ENCRYPTED\nDEK-Info: AES-256-CBC,6EF2C95AF143341584114A13ED0A8C06\n\nXqSK8L2MLJBHATHIslIYqWklO9nncVgt/vy0tntlNkVafuMs7PKzL9bHvJU5w2UG\n1inznGkrNzCjkjN3RLQ0Cd2uWQPOw/zyk6HH66ORXxmQtubKq9gkZwo7IHd+TAZJ\nsbmYYSO+DeKFPSmjOS5srr9wulBz4dggLiiVwPx1PCoMiHePDW63Tt1wtdW6TNWL\na4tZ/GF3YBoNJjHLCwmTL0ueaiUfw2/uNKpgUUAN/3tDOlFmoO/lRbil/IF+pcH3\nETy+ShVULuwHzSEkt+73NdwcR5z7k5DgYA0yZvqwm76KtVsqTZ8VuueIxU9zpwRw\ncH5SqkhoY6e3xgzTqWtgwkm+QFozweondAj2LicQocbw9CEFmZQeAYHjMSWdScXV\n/ZRsjpZPhsmaIN9y0HqJMigF1tXxR1I00+OxPAxyaR5qKB+uauRtJzy6GdanTj1A\nO+AczdN3vO0mwQBSdKK6BCYNzaaq6w9jJM1kvYSMYMWQP7MIJYW2e9165URvgvcy\nnLc2ePM3bsLVY2K56cemlYEXOWRsnwFGHXnGvwiI222tsJ+Ng+v1VsV5FzIe67aX\no9HOY8hvDlSFB/zAlUGHB5Vxw2K340osIw+Nsec5y1JT6Oive6O9AY6B/r+up4s+\nRxccDFGmtbHUSBlp/a2pRM3sPDNYcYgMJfTRlupTl4QA3eOndMyeCWHbaePD0m/5\nIr79R8PIBPP1T1sV1RrpBZcGO/metQUIdhCJMrHggFOqT6jhR1Xe/DvEHsm1yd90\nwkUHSTwM3eHo5O9B/vBlyguF0hzK/6DRK9xhMbpl+QTuYGAgeHFrldGA8jBunOV3\neG1yUrhUdyk5vik4bQyEvzCupWy5QqR+AqEbiUvaTY6K3nGZ7d8edwGCgZQx4MTR\nzii54EvfpVMympLTB1FzmYxfF/KPET1zCi3BwTtMmFBh2qFx0d0mnoJRAUvxCYU7\nsMHpuyot2N7bdkWVyGqwfaYhk4HupJq/5ShaZPzEO9+Krsn3GgIvMRSgyucMYMPt\n2pk8Z8fAGboV9wGZi0eG5xWgBdN0Ri0l75byeXRgFy4NjWSB9PIZxpXJqinqb+07\n2hvMYj4YcEM0Xs2cZDCFxLMrCCpu3k6NwtbC2Yj3XwS3AlW1FlVw1ap7zjHyw/Fy\nUOQA+ayctJ3vZrax/UU/IEaJLK6aCAkpHVDoEtVsKu5oFpyLBeduBq+AZo8gabsJ\n2vgSYX0Qde4CIOy/2zaXwMbhbupLt4Q4JlcM7kkV6UDPE7FYVx2VP+iCphjGZRo4\nCBgfGGnZCm2vUwzYEaaEWBVupKR49GzsXmCHoli+SYxvbpnEnXfPk3HfYmAzqPLL\nyFzmbA2/KJDdsjQzspAOUjDsrWkSMRA6Qz/jf62LUgOY+75dq52tMcZgaqJbhKsH\nuDyqpxBiySlLfAtZjyLohPnD8pZIfvHTvfTaipzgeXhAq9yaXNLiRT4Z2OPByfeE\nkZkbn7YgU0DtDretdOdl6+Pw9yFwWTwA++1Xp8CHYeYBvQyWUYfTCKCRRVEXT/sN\nW2VKJXSSsa5SJ3ZV8c8cDsXO+b0xAwDL+k6cc09mGauvHqV8AGHPsP2iacV+ZRtT\nB/6h8ZNoJKPXwc1bgm+dg65WOn0b5BFgcMSRGaEmttn8C36nTFthiL9AtrmYKmjv\ny0kL4KWPupguqzG1dpVgpOy++C0gF3+jCUirgVDwZRUY8lgTYa1l+Ko903R2e06j\npfC4ASJPdB+pwCrEJwaEfCHPhzeEMVl5SfRzpquIjBjYNFaQ/tzYVm/EUZiiCVGu\nABJHV6u+khj4bWXho5+STNkbZ4bKpZcnBCf7xVV3ckSg8158EOIbSEFaVC3JiNBH\nf4yN24x5gqFQm3UfmeDm15QT54RiISxKADmfEFqirtIGxPTzm4f4xH9tH6KrcPH8\n3Ak0dn4bxCE6/VGTNG9KmxkiIKP8OKjVdehXxhcqesfrhFfNUfOoSvzBWsLMa+IL\ngul9/QsFnd69Xu8YQZSAIrlqJZd8GA9J9LgVC2VurqELBBN2lUiPinp8vjmTQLAa\nXzFvP7zPd5TKubFA+8W9ASHRplMdbtP1ej8CWmRO37eqamq45hsiONq9hsg1ngbA\nFGyaP4lmuCpqg0jog7c8tZjJhLfySK0VzbxDOem3WmRE/eamzoOE+f9vSUJXjBZY\noP7kuEvbllmPnkxQi6YkWUgs/ovpH2WdKFm1vsci4xoNfLCS19L8AUVqUW4BeEqG\nOs4mgp98bxPHHxmdWNtFPotGSn2JC7uyZaL/6EkaUvnWc9vogZzwMiLo0gXmtWDn\nwj+qvGOISzVJ5E0CivpEkRsFxuSVgYL7lfwmbKanEZyqm26/bDaFm3Ie+jNy4z+F\n-----END RSA PRIVATE KEY-----"

# Parse and decrypt successfully with valid password
load_pem_private_key(encrypted_rsa_key, b"pw")

# Fail with invalid password and expected error message: "Bad decrypt. Incorrect password?"
load_pem_private_key(encrypted_rsa_key, b"not pw")

# Fail with different invalid password and unexpected error message: "Could not deserialize key data..."
load_pem_private_key(encrypted_rsa_key, b"bad pw")

[jku]

yes, I see the same errors

[lukpueh]

pyca/cryptography#8563

[jku]

Let's assume this is fixed with #546, reopen if not