Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[🚀 Feature]: webdriver connect pool maxsize increase #14594

Open
Liu-XinYuan opened this issue Oct 14, 2024 · 3 comments
Open

[🚀 Feature]: webdriver connect pool maxsize increase #14594

Liu-XinYuan opened this issue Oct 14, 2024 · 3 comments

Comments

@Liu-XinYuan
Copy link

Feature and motivation

When I am doing automated testing, I need to asynchronously monitor whether the verification code pop-up box appears. At this time, because the default connection pool size is 1, performance queue and log warnings appear.
[Warning] The connection pool is full and connections are being dropped: xxxx. Connection pool size: 1

Usage example

Asynchronously monitor pop-up boxes that appear randomly

Copy link

@Liu-XinYuan, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

@Liu-XinYuan
Copy link
Author

Liu-XinYuan commented Oct 14, 2024

The code involved is as follows:
class RemoteConnection

def _get_connection_manager(self):
    pool_manager_init_args = {"timeout": self.get_timeout()}
    if self._ca_certs:
        pool_manager_init_args["cert_reqs"] = "CERT_REQUIRED"
        pool_manager_init_args["ca_certs"] = self._ca_certs

    if self._proxy_url:
        if self._proxy_url.lower().startswith("sock"):
            from urllib3.contrib.socks import SOCKSProxyManager

            return SOCKSProxyManager(self._proxy_url, **pool_manager_init_args)
        if self._identify_http_proxy_auth():
            self._proxy_url, self._basic_proxy_auth = self._separate_http_proxy_auth()
            pool_manager_init_args["proxy_headers"] = urllib3.make_headers(proxy_basic_auth=self._basic_proxy_auth)
        return urllib3.ProxyManager(self._proxy_url, **pool_manager_init_args)

    return urllib3.PoolManager(**pool_manager_init_args)``

@diemol diemol added C-py and removed needs-triaging labels Nov 5, 2024
@navin772
Copy link
Contributor

Hi @Liu-XinYuan, you can increase the poolsize via init_args_for_pool_manager from the Client config, this should work for both direct connections and proxy connections:

from selenium import webdriver
from selenium.webdriver.remote.client_config import ClientConfig

client_config = ClientConfig(
    remote_server_addr="http://ip_addr:port",

    init_args_for_pool_manager={
        "init_args_for_pool_manager": {
            "maxsize": 20,  # Set pool size
        }
    }
)

driver = webdriver.Remote(client_config=client_config)

Client config was introduced in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants