authlib 0.15.1 does not send authorizaton header

[btel]

Describe the bug

authlib 0.15.1 does not seem to send authentication token with the request

Error Stacks

with authlib 0.15.1

RACE [2020-10-15 15:08:46] httpx._config - load_verify_locations cafile=/home/bartosz/.pyenv/versions/3.8.3/envs/quetz-heroku-test/lib/python3.8/site-packages/certifi/cacert.pem
TRACE [2020-10-15 15:08:46] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'api.github.com', 443)
TRACE [2020-10-15 15:08:46] httpcore._async.connection_pool - created connection=<AsyncHTTPConnection http_version=UNKNOWN state=0>
TRACE [2020-10-15 15:08:46] httpcore._async.connection_pool - adding connection to pool=<AsyncHTTPConnection http_version=UNKNOWN state=0>
TRACE [2020-10-15 15:08:46] httpcore._async.connection - open_socket origin=(b'https', b'api.github.com', 443) timeout={'connect': 5.0, 'read': 5.0, 'write': 5.0, 'pool': 5.0}
TRACE [2020-10-15 15:08:46] httpcore._async.connection - create_connection socket=<httpcore._backends.asyncio.SocketStream object at 0x7fa23a78e1c0> http_version='HTTP/1.1'
TRACE [2020-10-15 15:08:46] httpcore._async.connection - connection.arequest method=b'GET' url=(b'https', b'api.github.com', None, b'/user') headers=[(b'Host', b'api.github.com'), (b'Accept', b'*/*'), (b'Accept-Encoding', b'gzip, deflate'), (b'Connection', b'keep-alive'), (b'User-Agent', b'Authlib/0.15.1 (+https://authlib.org/)')]
TRACE [2020-10-15 15:08:46] httpcore._async.http11 - send_request method=b'GET' url=(b'https', b'api.github.com', None, b'/user') headers=[(b'Host', b'api.github.com'), (b'Accept', b'*/*'), (b'Accept-Encoding', b'gzip, deflate'), (b'Connection', b'keep-alive'), (b'User-Agent', b'Authlib/0.15.1 (+https://authlib.org/)')]
TRACE [2020-10-15 15:08:46] httpcore._async.http11 - send_data=Data(<0 bytes>)
DEBUG [2020-10-15 15:08:47] httpx._client - HTTP Request: GET https://api.github.com/user "HTTP/1.1 401 Unauthorized"
TRACE [2020-10-15 15:08:47] httpcore._async.http11 - receive_event=Data(<131 bytes>)
TRACE [2020-10-15 15:08:47] httpcore._async.http11 - receive_event=EndOfMessage(headers=<Headers([])>)
TRACE [2020-10-15 15:08:47] httpcore._async.http11 - response_closed our_state=DONE their_state=DONE

with authlib 0.14.3 it's ok (the code was not changed otherwise)

TRACE [2020-10-15 15:08:02] httpx._config - load_ssl_context verify=True cert=None trust_env=True http2=False
TRACE [2020-10-15 15:08:02] httpx._config - load_verify_locations cafile=/home/bartosz/.pyenv/versions/3.8.3/envs/quetz-heroku-test/lib/python3.8/site-packages/certifi/cacert.pem
TRACE [2020-10-15 15:08:02] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'api.github.com', 443)
TRACE [2020-10-15 15:08:02] httpcore._async.connection_pool - created connection=<AsyncHTTPConnection http_version=UNKNOWN state=0>
TRACE [2020-10-15 15:08:02] httpcore._async.connection_pool - adding connection to pool=<AsyncHTTPConnection http_version=UNKNOWN state=0>
TRACE [2020-10-15 15:08:02] httpcore._async.connection - open_socket origin=(b'https', b'api.github.com', 443) timeout={'connect': 5.0, 'read': 5.0, 'write': 5.0, 'pool': 5.0}
TRACE [2020-10-15 15:08:02] httpcore._async.connection - create_connection socket=<httpcore._backends.asyncio.SocketStream object at 0x7f6d43dd3850> http_version='HTTP/1.1'
TRACE [2020-10-15 15:08:02] httpcore._async.connection - connection.arequest method=b'GET' url=(b'https', b'api.github.com', None, b'/user') headers=[(b'Host', b'api.github.com'), (b'Accept', b'*/*'), (b'Accept-Encoding', b'gzip, deflate'), (b'Connection', b'keep-alive'), (b'User-Agent', b'Authlib/0.14.3 (+https://authlib.org/)'), (b'Authorization', b'Bearer 706db6c985d50bc04fb61c27e0c3a327b966d9d0')]
TRACE [2020-10-15 15:08:02] httpcore._async.http11 - send_request method=b'GET' url=(b'https', b'api.github.com', None, b'/user') headers=[(b'Host', b'api.github.com'), (b'Accept', b'*/*'), (b'Accept-Encoding', b'gzip, deflate'), (b'Connection', b'keep-alive'), (b'User-Agent', b'Authlib/0.14.3 (+https://authlib.org/)'), (b'Authorization', b'Bearer 706db6c985d50bc04fb61c27e0c3a327b966d9d0')]
TRACE [2020-10-15 15:08:02] httpcore._async.http11 - send_data=Data(<0 bytes>)
DEBUG [2020-10-15 15:08:02] httpx._client - HTTP Request: GET https://api.github.com/user "HTTP/1.1 200 OK"
TRACE [2020-10-15 15:08:02] httpcore._async.http11 - receive_event=Data(<517 bytes>)
TRACE [2020-10-15 15:08:02] httpcore._async.http11 - receive_event=EndOfMessage(headers=<Headers([])>)
TRACE [2020-10-15 15:08:02] httpcore._async.http11 - response_closed our_state=DONE their_state=DONE

To Reproduce

this is a piece of code that triggered this issue:

from authlib.integrations.starlette_client import OAuth
oauth = OAuth()
oauth.register(
        name='github',
        client_id=config.github_client_id,
        client_secret=config.github_client_secret,
        access_token_url='https://github.com/login/oauth/access_token',
        access_token_params=None,
        authorize_url='https://github.com/login/oauth/authorize',
        authorize_params=None,
        api_base_url='https://api.github.com/',
        client_kwargs={'scope': 'user:email'},
        quetz_db_url=config.sqlalchemy_database_url,
    )
router.route('/auth/github/authorize', name='authorize_github')
async def authorize(request: Request):
    token = await oauth.github.authorize_access_token(request)
    print(token)
    resp = await oauth.github.get('user', token=token)

Expected behavior

authentication token is sent and the response from api.github.com/user returns with code 200

Environment:

• OS: linux
• Python Version: 3.8.3
• Authlib Version: 0.15.1

Additional context

Add any other context about the problem here.

[firesock]

Another reproduction:

from unittest.mock import Mock

import httpx._config
import pytest
from authlib.integrations.starlette_client import OAuth
from authlib.integrations.httpx_client.oauth2_client import AsyncClient, OAuth2Auth

oauth = OAuth()
oauth.register(
    name="sso",
    client_id="id",
    client_secret="secret",
    api_base_url="https://api.example.com/",
    authorize_url="https://example.com/oauth/authorize",
    access_token_url="https://example.com/oauth/access_token",
    userinfo_endpoint="https://example.com/openid/userinfo",
)

class InterruptRequest(Exception):
    pass

token = {
    "token_type": "Bearer",
    "expires_in": 30000,
    "access_token": "ACCESS_TOKEN",
    "refresh_token": "REFRESH_TOKEN",
}

@pytest.mark.asyncio
async def test_userinfo_ignores_token(monkeypatch):
    # Prevent resolving example.com
    request_mock = Mock(side_effect=InterruptRequest())
    AsyncClient.request = request_mock

    with pytest.raises(InterruptRequest):
        await oauth.sso.userinfo(token=token)

    assert request_mock.call_count == 1
    assert request_mock.call_args.kwargs["auth"] is not None


@pytest.mark.asyncio
async def test_userinfo_accepts_token(monkeypatch):
    # Prevent resolving example.com
    request_mock = Mock(side_effect=InterruptRequest())
    AsyncClient.request = request_mock

    with pytest.raises(InterruptRequest):
        await oauth.sso.userinfo(token=token, auth=httpx._config.UNSET)

    assert request_mock.call_count == 1
    assert request_mock.call_args.kwargs["auth"] is not None

I've intercepted the call to httpx's AsyncClient and it appears that we're not passing an auth kwarg when token is provided. The problem appears to be caused by the mismatch between the auth arg check here https://github.com/lepture/authlib/blob/master/authlib/integrations/httpx_client/oauth2_client.py#L85 and the default argument in the method (None instead of httpx._config.UNSET).

I'm not sure of the intended behaviour here, but it makes sense to convert None -> UNSET if necessary? Worked around it in the second test by passing UNSET as the auth kwarg to my immediate call.

Noticed in:
Python 3.8.6
authlib 0.15.1
httpx 0.16.1

[lepture]

Please lock your httpx to 0.14.x. httpx has breaking change again.

[lepture]

It is caused by 0.16.0:

Preserve HTTP header casing.

[btel]

Thanks @lepture , I tested the code in the description with httpx==0.14.1 with the same results (no auth header for authlib 0.15.1).

The test by @firesock do not pass either when I install authlib 0.15.1 and httpx 0.14.1.

My environment:

attrs==20.2.0
Authlib==0.15.1
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
cryptography==3.1.1
h11==0.9.0
httpcore==0.10.2
httpx==0.14.1
idna==2.10
iniconfig==1.1.1
packaging==20.4
pluggy==0.13.1
py==1.9.0
pycparser==2.20
pyparsing==2.4.7
pytest==6.1.1
pytest-asyncio==0.14.0
rfc3986==1.4.0
six==1.15.0
sniffio==1.2.0
starlette==0.13.8
toml==0.10.1

results of the test:

test_authlib.py::test_userinfo_ignores_token FAILED                                                                                                                                                          [ 50%]
test_authlib.py::test_userinfo_accepts_token PASSED                                                                                                                                                          [100%]

===================================================================================================== FAILURES =====================================================================================================
___________________________________________________________________________________________ test_userinfo_ignores_token ____________________________________________________________________________________________

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f0bdb7b16d0>

    @pytest.mark.asyncio
    async def test_userinfo_ignores_token(monkeypatch):
        # Prevent resolving example.com
        request_mock = Mock(side_effect=InterruptRequest())
        AsyncClient.request = request_mock
    
        with pytest.raises(InterruptRequest):
            await oauth.sso.userinfo(token=token)
    
        assert request_mock.call_count == 1
>       assert request_mock.call_args.kwargs["auth"] is not None
E       assert None is not None

both tests pass after downgrading authlib to 0.14.3

[firesock]

It is caused by 0.16.0:

Preserve HTTP header casing.

Thanks for following up @lepture. Is this correct though? httpx fixed their issue with casing in encode/httpx#1351 which was released with 0.16.1, which works with the work around for me.

When I followed up more by examining the request going out, the Authorization header wasn't there at all, not even in the wrong case without the work around. @btel's original trace shows the same behaviour.

[lepture]

@btel @firesock can you try with the latest maintain-0.15 code? I think it fixed the problem.

[firesock]

Thanks @lepture, that branch fixes it!

[btel]

the fix in maintain-0.15 branch also fixes my code. Thanks for following up and fixing it @lepture @firesock

[lepture]

0.15.2 released.