Implement HTTPS Proxy Support

[cooperlees]

Long story short

Once TLS in TLS support is added to asyncio (https://bugs.python.org/issue37179) is landed to (hopefully) Python 3.8 we should plan including support for HTTPS Proxy servers if Python >= 3.8.

Expected behaviour

Have the ability to use no authentication HTTPS proxies and Authentication via using custom SSL Contexts: Example Code

Actual behaviour

Today we get explicitly told aiohttp can't do HTTPS proxies:

[cooper:~:( (aiohttp_auth_proxy))]$ python3.7 aioclient.py
HTTPS proxies https://proxy:1443 are not supported, ignoring

Steps to reproduce

Set via the environment or other means to use an HTTPS proxy.

#! /usr/bin/env python3
 
import asyncio
import logging
import os
import ssl
import sys
import time
 
import aiohttp
 
 
PROXY_PORT = 1443
PROXY_HOSTNAME = "https://proxy.company.com"
EXTERNAL_ENDPOINT = "https://www.google.com"
CA_BUNDLE = "/var/certs/ca.pem"
 
 
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
logger.addHandler(logging.StreamHandler(sys.stdout))
 
 
async def run_example():
    cert = os.environ["TLS_CL_CERT_PATH"]
    key = os.environ["TLS_CL_KEY_PATH"]
 
    if not cert or not key:
        raise ValueError("Missing key TLS cert settings.")
 
    # For the example lets ensure HTTPS_PROXY is set
    os.environ["HTTPS_PROXY"] = f"{PROXY_HOSTNAME}:{PROXY_PORT}"
 
    # Setup SSL Fun
    ssl_ctx = ssl.create_default_context(cafile=CA_BUNDLE)
    ssl_ctx.load_cert_chain(cert, key)
    conn = aiohttp.TCPConnector(ssl=ssl_ctx)
 
    start_time = time.time()
    # trust_env allows HTTP(s)_PROXY vars to work
    async with aiohttp.ClientSession(connector=conn, trust_env=True) as session:
        async with session.get(EXTERNAL_ENDPOINT) as response:
            logger.info(
                "Received response with status code "
                + f"{response.status} in {time.time() - start_time}s"
            )
 
 
if __name__ == "__main__":
    asyncio.run(run_example())

Your environment

• Python 3.7.3
• aiohttp 3.5.4

[cooperlees]

@1st1 has stated that this should work with uvloop. Would a PR be accepted that allow's uvloop enabled programs to work until we get the stdlib asyncio loop to have working TLS in TLS?

[webknjaz]

I think so