certificate is not valid for IP #5110

lilydjwg · 2020-10-24T11:37:52Z

🐞 Describe the bug

When requesting with IPv6 enabled, I get this weird exception.

💡 To Reproduce

python3 -m venv venv and . venv/bin/activate
pip install -U aiohttp
Run code:

import asyncio
import aiohttp

async def main():
  async with aiohttp.ClientSession() as session:
    r = await session.get('https://pypi.org/')
    print(await r.text())

if __name__ == '__main__':
  asyncio.run(main())

See error.

💡 Expected behavior

📋 Logs/tracebacks

Traceback (most recent call last): 
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 946, in _wrap_create_connection  
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore  # noqa
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1050, in create_connection                                      
    transport, protocol = await self._create_connection_transport(
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1080, in _create_connection_transport                                              await waiter                                                                                                                          File "/usr/lib/python3.8/asyncio/sslproto.py", line 529, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/lib/python3.8/asyncio/sslproto.py", line 189, in feed_ssldata                                                             
    self._sslobj.do_handshake()
  File "/usr/lib/python3.8/ssl.py", line 944, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid 
for '151.101.128.223'. (_ssl.c:1124)

The above exception was the direct cause of the following exception: 

Traceback (most recent call last):
  File "t.py", line 12, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "t.py", line 8, in main
    r = await session.get('https://pypi.org/')
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/client.py", line 490, in _request
    conn = await self._connector.connect(
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 528, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 868, in _create_connection
    _, proto = await self._create_direct_connection(
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 1023, in _create_direct_connection
    raise last_exc
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 999, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "/home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 948, in _wrap_create_connection
    raise ClientConnectorCertificateError(
aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host pypi.org:443 ssl:True [SSLCertVerificationError: (1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '151.101.128.223'. (_ssl.c:1124)")]

📋 Your version of the Python

$ python --version
Python 3.8.6

📋 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.7.0
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: [email protected]
License: Apache 2
Location: /home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages
Requires: yarl, attrs, async-timeout, chardet, multidict
Required-by:

$ python -m pip show multidict
Name: multidict
Version: 5.0.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages
Requires: 
Required-by: yarl, aiohttp

$ python -m pip show yarl
Name: yarl
Version: 1.6.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /home/lilydjwg/tmpfs/venv/lib/python3.8/site-packages
Requires: idna, multidict
Required-by: aiohttp

📋 Additional context

I find this when I'm trying to reproduce another error.

The text was updated successfully, but these errors were encountered:

djmitche · 2020-10-24T15:41:02Z

Seeing the same with python 3.6 and 3.7.

Logs are here and here.

It looks like aiohttp is passing an IP to the SSL verification.

djmitche · 2020-10-24T15:45:03Z

Repro for python 3.6:

import aiohttp

loop = asyncio.get_event_loop()

async def main():
  async with aiohttp.ClientSession(loop=loop) as session:                                                                                                                                                                                                                                                                                                                                      
    r = await session.get('https://pypi.org/')
    print(await r.text())

if __name__ == '__main__':
  loop.run_until_complete(main())

djmitche · 2020-10-24T15:52:12Z

In aiohttp/connector.py it looks like hinfo['hostname'] is a name in aiohttp==3.6.3 and an IP in aiohttp==3.7.0.

djmitche · 2020-10-24T16:00:09Z

In the good case, self._resolver.resolve (a ThreadedResolver) returns

resolver -> [{'hostname': 'pypi.org', 'host': '151.101.192.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '151.101.128.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '151.101.64.223', 'po rt': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '151.101.0.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '2a04:4e42::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '2a04:4e42:600::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org', 'host': '2a04:4e42:400::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'pypi.org ', 'host': '2a04:4e42:200::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}]

In the bad case,

[{'hostname': '151.101.192.223', 'host': '151.101.192.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '151.101.128.223', 'host': '151.101.128.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '15 1.101.64.223', 'host': '151.101.64.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '151.101.0.223', 'host': '151.101.0.223', 'port': 443, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '2a04:4e42::223', 'host': '2a04:4e42::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '2a04:4e42:600::223', 'host': '2a04:4e42:600::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '2a04:4e42:400::223', 'host': '2a04 :4e42:400::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}, {'hostname': '2a04:4e42:200::223', 'host': '2a04:4e42:200::223', 'port': 443, 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICSERV|AI_NUMERICHOST: 1028>}]

djmitche · 2020-10-24T16:03:07Z

Old resolve:

aiohttp/aiohttp/resolver.py

Lines 27 to 40 in 1940503

    
           async def resolve(self, host: str, port: int=0, 
        
                             family: int=socket.AF_INET) -> List[Dict[str, Any]]: 
        
               infos = await self._loop.getaddrinfo( 
        
                   host, port, type=socket.SOCK_STREAM, family=family) 
        
               hosts = [] 
        
               for family, _, proto, _, address in infos: 
        
                   hosts.append( 
        
                       {'hostname': host, 
        
                        'host': address[0], 'port': address[1], 
        
                        'family': family, 'proto': proto, 
        
                        'flags': socket.AI_NUMERICHOST}) 
        
               return hosts

New resolve:

aiohttp/aiohttp/resolver.py

Lines 28 to 53 in cac3cd2

    
           async def resolve(self, host: str, port: int=0, 
        
                             family: int=socket.AF_INET) -> List[Dict[str, Any]]: 
        
               infos = await self._loop.getaddrinfo( 
        
                   host, port, type=socket.SOCK_STREAM, family=family) 
        
               hosts = [] 
        
               for family, _, proto, _, address in infos: 
        
                   if family == socket.AF_INET6 and address[3]:  # type: ignore 
        
                       # This is essential for link-local IPv6 addresses. 
        
                       # LL IPv6 is a VERY rare case. Strictly speaking, we should use 
        
                       # getnameinfo() unconditionally, but performance makes sense. 
        
                       host, _port = socket.getnameinfo( 
        
                           address, socket.NI_NUMERICHOST | socket.NI_NUMERICSERV) 
        
                       port = int(_port) 
        
                   else: 
        
                       host, port = address[:2] 
        
                   hosts.append({ 
        
                       'hostname': host, 
        
                       'host': host, 
        
                       'port': port, 
        
                       'family': family, 
        
                       'proto': proto, 
        
                       'flags': socket.AI_NUMERICHOST | socket.AI_NUMERICSERV, 
        
                   }) 
        
               return hosts

so it looks like this is a simple matter of variable shadowing. However, it's a very serious error!

aio-libs/aiohttp#5110 means that https URLs don't work. aio-libs/aiohttp#5118 fixes the issue, and will be included in 3.7.1.

Issue: aio-libs/aiohttp#5110

lilydjwg added the bug label Oct 24, 2020

lilydjwg mentioned this issue Oct 24, 2020

Network is unreachable with IPv6 enabled sites and no local IPv6 route #5112

Closed

This was referenced Oct 24, 2020

Python client tests failing with certificate error taskcluster/taskcluster#3767

Closed

Return hostnames from ThreadedResolver #5118

Merged

asvetlov closed this as completed in #5118 Oct 24, 2020

bdraco mentioned this issue Oct 24, 2020

Upgrade to aiohttp 3.7.1 home-assistant/core#42305

Merged

21 tasks

djmitche added a commit to djmitche/taskcluster that referenced this issue Oct 25, 2020

Don't use aiohttp==3.7.0

035960f

aio-libs/aiohttp#5110 means that https URLs don't work. aio-libs/aiohttp#5118 fixes the issue, and will be included in 3.7.1.

markusressel mentioned this issue Oct 25, 2020

Connect to Telegram server is Error aiogram/aiogram#448

Closed

jiri-janousek added a commit to tiliado/flatpak-manifest-update that referenced this issue Oct 25, 2020

Pin aiohttp[speedups] ==3.6.*

2d1dadb

Issue: aio-libs/aiohttp#5110

This was referenced Oct 25, 2020

Improve error reporting from openapi calls pulp/pulp-cli#21

Merged

Pin some requirements pulp/pulpcore#984

Merged

Harmon758 mentioned this issue Nov 22, 2020

Cannot run discord bot, with SSL error. Rapptz/discord.py#5967

Closed

gozdal mentioned this issue Jan 6, 2022

Failure establishing connection when IPv6 network is unreachable while IPv4 works #6496

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

certificate is not valid for IP #5110

certificate is not valid for IP #5110

lilydjwg commented Oct 24, 2020 •

edited

Loading

djmitche commented Oct 24, 2020

djmitche commented Oct 24, 2020

djmitche commented Oct 24, 2020

djmitche commented Oct 24, 2020

djmitche commented Oct 24, 2020

certificate is not valid for IP #5110

certificate is not valid for IP #5110

Comments

lilydjwg commented Oct 24, 2020 • edited Loading

djmitche commented Oct 24, 2020

djmitche commented Oct 24, 2020

djmitche commented Oct 24, 2020

djmitche commented Oct 24, 2020

djmitche commented Oct 24, 2020

lilydjwg commented Oct 24, 2020 •

edited

Loading