Same input causes exception in python 3.8.3 but not in python 3.6.8

[juergen-kaiser-by]

Describe the bug

An input for authlib.jose.jwt.decode() causes an exception on python 3.8.3 while it works on python 3.6.8 .

I'm using:

from authlib.jose import jwt
jwt.decode(mytoken, key=mykey, claims_options=something)

The causing parameter is key. In my case, the (shortened) value is :

{
  "keys": [
    {
      "kid": "something2",
      "kty": "RSA",
      "alg": "RS256",
      "use": "sig",
      "n": "something3",
      "e": "AQAB",
      "x5c": ["something4"],
      "x5t": "something5",
      "x5t#S256": "something6"
    }
  ]
}

However, if I set mykey to the key in the inner list:

{
      "kid": "something2",
      "kty": "RSA",
      "alg": "RS256",
      "use": "sig",
      "n": "something3",
      "e": "AQAB",
      "x5c": ["something4"],
      "x5t": "something5",
      "x5t#S256": "something6"
    }

then authlib accepts the input for both, python 3.6.8 and 3.8.3.

Error Stacks

[...]
  File "/Users/njjn/.pyenv/versions/demo2/lib/python3.8/site-packages/authlib/jose/rfc7519/jwt.py", line 98, in decode
    data = self._jws.deserialize_compact(s, load_key, decode_payload)
  File "/Users/njjn/.pyenv/versions/demo2/lib/python3.8/site-packages/authlib/jose/rfc7515/jws.py", line 102, in deserialize_compact
    algorithm, key = self._prepare_algorithm_key(jws_header, payload, key)
  File "/Users/njjn/.pyenv/versions/demo2/lib/python3.8/site-packages/authlib/jose/rfc7515/jws.py", line 258, in _prepare_algorithm_key
    key = algorithm.prepare_key(key)
  File "/Users/njjn/.pyenv/versions/demo2/lib/python3.8/site-packages/authlib/jose/rfc7518/_cryptography_backends/_jws.py", line 41, in prepare_key
    return RSAKey.import_key(raw_data)
  File "/Users/njjn/.pyenv/versions/demo2/lib/python3.8/site-packages/authlib/jose/rfc7518/_cryptography_backends/_keys.py", line 116, in import_key
    return import_key(
  File "/Users/njjn/.pyenv/versions/demo2/lib/python3.8/site-packages/authlib/jose/rfc7518/_cryptography_backends/_keys.py", line 277, in import_key
    cls.check_required_fields(raw)
  File "/Users/njjn/.pyenv/versions/demo2/lib/python3.8/site-packages/authlib/jose/rfc7517/models.py", line 120, in check_required_fields
    raise ValueError('Missing required field: "{}"'.format(k))
ValueError: Missing required field: "e"

To Reproduce

(untested): get a valid key and put it in the structure as depicted in my example above.

Expected behavior

The same input gets accepted without errors for both python versions.

Environment:

• OS: macos/linux
• Python Version: 3.8.3
• Authlib Version: v0.15

[lepture]

Can you give a full example? With runnable code to reproduce your problem? I can't reproduce it with your given key.

[lepture]

I've rechecked the code. It would never happen with your given key, because the given key already contains "e". I guess there must be something wrong with your provided data.

[walgula]

I have the same error

[lepture]

@walgula is it possible for you to provide a runnable test case?

[walgula]

{
"keys": [
{
"kty": "RSA",
"kid": "hash;)",
"use": "sig",
"n": "some very long hash;)",
"e": "AQAB"
}
]
}

[walgula]

in 0.14.3 there is no error

[bjmc]

@walgula Sorry, but that's just a key, not a complete runnable test case. Can you write a short section of python that will produce the error when somebody runs it?

[dani]

Looks like I'm hitting the same issue. In my case it's running a Synapse server v1.21.0 (Matrix homeserver) with OIDC auth against a Lemonldap::NG issuer, with Py 3.6.8. Going back to authlib 0.14.3 makes everything working again.
Unfortunately, I'm really unsure how to provide a simple, isolated test case.
For reference, synapse's OIDC handler is https://github.com/matrix-org/synapse/blob/develop/synapse/handlers/oidc_handler.py

[lepture]

I may know the problem now. There is a refactor on Json Web Key Set.

cc @dani try to change load_jwks with:

from authlib.jose import JsonWebKey

JsonWebKey.import_key_set({"keys": [....]})

[dani]

Mmmmh, I'm afraid my very limited python-foo will ask for a bit more help ;-)
At which point in the load_jwks function should I add the JsonWebKey.import_key_set call ?

[lepture]

@dani @juergen-kaiser-by can you try with the code in maintain-0.15 branch?

[dani]

Updating authlib to the latest from maintain-0.15 branch fixes it for me. Synapse can auth against OIDC again ! :-)