Can't set proxy per request

[HMaker]

aiohttp allows one to set proxies per request, this is not possible with aiohttp_socks because it overrides session's connector which takes static proxy, so same proxy will be used for all requests of that session.

From the readme I understood this feature existed on aiosocksy but was removed to make codebase more maintainable, but it kills use cases which requires proxy rotation or proxy bypass.

If maintanability is a possible issue why not keep both versions? one subclass of TCPConnector for static proxies and another one for dynamic proxies. When the dynamic version break people can still use the static one. Also to avoid breakage this package can pin aiohttp version on requirements.txt.

[romis2012]

To send requests through a proxy, you first need to establish a connection with it. The connection to the proxy is performed by aiohttp.Session (via its connector).

aiohttp allows one to set proxies per request

In this case (proxy per request) aiohttp establishes a new connection to the proxy for each request internally (inside its TCPConnector), we do this explicitly via ProxyConnector

...this feature existed on aiosocksy...

It did it as described above in relation to aiohttp. This was not a good design

In any case, we can not modify aiohttp request methods (session.get(...) etc.) (and there is absolutely no need for this).

Why not use a function like this:

async def fetch_via_proxy(proxy_url, target_url):
    connector = ProxyConnector.from_url(proxy_url)
    async with aiohttp.ClientSession(connector=connector) as session:
        async with session.get(target_url) as res:
            return await res.text()

This is what you need if you use different proxies for each request.

[HMaker]

aiohttp creates new connections only when there are not any available in connections pool, even for proxies (they include proxy in ConnectionKey). Due connection pooling aiohttp suggests you to use a single session for entire application:

Unless you are connecting to a large, unknown number of different servers over the lifetime of your application, it is suggested you use a single session for the lifetime of your application to benefit from connection pooling.

Proxy per request could be easily implemented by overriding TCPConnector._create_proxy_connection() if aiohttp_socks could wrap an existing asyncio stream to perform tunnel creation thought CONNECT requests like aiohttp does for HTTP proxies.

[romis2012]

aiohttp creates new connections only when there are not any available in connections pool

And even If you send request to different origins or connect to different proxies? (or what do you mean by "proxy per request"?)

Due connection pooling aiohttp suggests you to use a single session for entire application

Yes, if you have a connection to the single server (in our case, a proxy server). But you want to change the proxy for every request, don't you?

Proxy per request could be easily implemented by overriding TCPConnector._create_proxy_connection()

This will require rewriting a lot of the TCPConnector code (via copy-pasting, how it was implemented aiosocksy)

It all makes no sense.

aiohttp session is just a connection to the proxy server, and for different proxies you must open a new session.

[HMaker]

I didn't mean unique proxy per request, but allow proxies to be changed arbitrarly for same session. Yes aiohttp will create new connection for different origins, the TCPConnector uses SSL context, server hostname and port, and proxy URL and authentication as key for created connection, this connection will be reused only when a new request matches this key.

Anyway, I will fork this project or reimplement socks5 client myself.

[romis2012]

Anyway, I will fork this project or reimplement socks5 client myself.

Good luck

[HMaker]

I implemented that and published now at https://github.com/HMaker/aiohttp-socks5

This will require rewriting a lot of the TCPConnector code (via copy-pasting, how it was implemented aiosocksy)

Not true, check implementation above.

[MrCoder1453]

I implemented that and published now at https://github.com/HMaker/aiohttp-socks5

This will require rewriting a lot of the TCPConnector code (via copy-pasting, how it was implemented aiosocksy)

Not true, check implementation above.

You're a king bro.