How to set authentication socks proxy on docker Selenium Grid

[Eeyhan]

Thank you very much for developing this library that is so useful ，I've used it in a formal project.

Now, I have a question，how to set authentication socks proxy on docker Selenium Grid?

I know of a Remote class，but I don't know how to configure it

my code :

# coding=utf-8
import time
from seleniumwire import webdriver

chrome_capabilities = {
    "browserName": "chrome",
    "version": "",
    "platform": "ANY",
    "javascriptEnabled": True,
    'applicationName': 'test1'
}

sw_options = {
    'addr': '0.0.0.0',
    'port': 8087,
    'auto_config': False,  # Ensure this is set to False
    'proxy': {
        'http': 'socks5://user:pass@proxy_ip:proxy_port',
        'https': 'socks5://user:pass@proxy_ip:proxy_port'
    }
}



# browser = webdriver.Remote("http://docker_ip:docker_host/wd/hub", desired_capabilities=chrome_capabilities)
browser = webdriver.Remote("http://docker_ip:docker_host/wd/hub", desired_capabilities=chrome_capabilities,
                           seleniumwire_options=sw_options)
browser.get("http://httpbin.org/get")
print(browser.title)
time.sleep(60)
browser.get_screenshot_as_file("8.png")
browser.quit()

it still shows the IP of the real machine and not proxy ip

I try this config：issue220，maybe he used Kubernetes ,and I use doker，so I guess it's a little different

I have been afraid of confusing you with unclear expressions, so I'm sorry that I edited it for several times

Looking forward to your reply

[wkeeling]

Thanks for raising this.

I think you're just missing the options to point the browser back at Selenium Wire so that it can capture requests. Without that, the browser will just go direct to the URL and the socks proxy won't take effect.

It should just be a case of adding the ChromeOptions to the webdriver instance, e.g.

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=<container_ip_running_sw>:8087')
chrome_options.add_argument('--ignore-certificate-errors')

browser = webdriver.Remote(
    "http://docker_ip:docker_host/wd/hub", 
    options=chrome_options,
    desired_capabilities=chrome_capabilities,
    seleniumwire_options=sw_options
)

Change <container_ip_running_sw> to the IP address of the container running Selenium Wire. This IP must be visible to the container running the browser.

[Eeyhan]

Thank you very much for your reply，and I've tried it that way，

here is my complete code:

# coding=utf-8
import time
from seleniumwire import webdriver

chrome_capabilities = {
    "browserName": "chrome",
    "version": "",
    "platform": "ANY",
    "javascriptEnabled": True,
    'applicationName': 'test1'
}

sw_options = {
    'addr': '0.0.0.0',
    'port': 8087,
    'auto_config': False,  # Ensure this is set to False
    'proxy': {
        'http': 'socks5://user:pass@proxy_ip:proxy_port',
        'https': 'socks5://user:pass@proxy_ip:proxy_port'
    }
}

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=container_ip_running_sw_ip:8087')
chrome_options.add_argument('--ignore-certificate-errors')

browser = webdriver.Remote(
    "http://docker_ip:docker_host/wd/hub",
    options=chrome_options,
    desired_capabilities=chrome_capabilities,
    seleniumwire_options=sw_options
)
browser.get("http://httpbin.org/get")
print(browser.title)
time.sleep(60)
browser.get_screenshot_as_file("8.png")
browser.quit()

It reports an error indicating that the agent is in error：

Traceback (most recent call last):
  File "E:/dd/utils/docker_selenium_test_wire.py", line 34, in <module>
    browser.get("http://httpbin.org/get")
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_PROXY_CONNECTION_FAILED
  (Session info: chrome=91.0.4472.77)

The httpbin.org site also did not display the results properly.

The above error appears to be an proxy problem，it's not a proxy problem, because I can call locally with Selenium，I thought it was the container host problem，but the hub and node of my Selenium Grid are on the same host，That is to say，The IP on which the Selenium Wire container is running is the same IP as the selenium grid hub .

So, I don't know what to do，sorry to bother you again

[wkeeling]

Ok thanks. Does it work if you don't use the upstream socks proxy - have you tried omitting the proxy from the sw_options? If that works, at least we know that the other options are good.

[Eeyhan]

Thanks,Let me try again how to configure it,by the way,this library is really good to use

[sreethal]

I want to run the testcases in parallel. Is there a way to generate different ports automatically . Because in the statement "chrome_options.add_argument('--proxy-server=container_ip_running_sw_ip:8087')" we are using the port 8087. If i want to execute the testcases parallel , how can we achieve.

[Eeyhan]

I want to run the testcases in parallel. Is there a way to generate different ports automatically . Because in the statement "chrome_options.add_argument('--proxy-server=container_ip_running_sw_ip:8087')" we are using the port 8087. If i want to execute the testcases parallel , how can we achieve.

I think it's definitely possible, but I don't know if the author @wkeeling has made any restrictions，If not, you can use a script to generate it。

example:

random.randint(81,65535)

before you do that, perhaps you should first determine which ports are not being used

[wkeeling]

@sreethal sorry for the delay in coming back. Yes you can run each instance of the webdriver in a separate thread. You can generate a list of port numbers in advance, and then pass a different port to each thread when you create it.

[wkeeling]

@Eeyhan is your original problem now resolved?

[sanjeevtrz]

@wkeeling am using similar configuration. Am running selenium grid, node and selenium wire in local machine.

   wire_options = {        
        'proxy': {
            'http': f'http://{username}:{password}@{host}:{port}',
            'https': f'httpS://{username}:{password}@{host}:{port}'
        },
        'suppress_connection_errors': False,
        'auto_config': False,
        'addr': '0.0.0.0',
        'port': 8087,
    }
    options = webdriver.ChromeOptions()
   
    options.add_argument("--proxy-server=localhost:8087")
    options.add_argument('--ignore-certificate-errors')
    proxy_driver = wiredriver.Remote(
        command_executor='http://localhost:4444/wd/hub',
        options=options,
        seleniumwire_options=wire_options)
   proxy_driver.get("https://api.ipify.org?format=json")

Failure is same..

Message: unknown error: net::ERR_PROXY_CONNECTION_FAILED

[4ndr01dHQD]

Faced the same problem, did anyone find a solution?

[zellsun]

When I use the real ip 192.168.0.xxx instead of 127.0.0.1 or 0.0.0.0, it works for me.

[saudBinHabib]

Thanks for raising this.

I think you're just missing the options to point the browser back at Selenium Wire so that it can capture requests. Without that, the browser will just go direct to the URL and the socks proxy won't take effect.

It should just be a case of adding the ChromeOptions to the webdriver instance, e.g.

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=<container_ip_running_sw>:8087')
chrome_options.add_argument('--ignore-certificate-errors')

browser = webdriver.Remote(
    "http://docker_ip:docker_host/wd/hub", 
    options=chrome_options,
    desired_capabilities=chrome_capabilities,
    seleniumwire_options=sw_options
)

Change <container_ip_running_sw> to the IP address of the container running Selenium Wire. This IP must be visible to the container running the browser.

Hi @wkeeling,

First of all, thank you for developing such an amazing package.

I am totally new to the selenium-wire, I am trying to connect remotely to selenium/hub chrome node or firefox node, I am able to connect if I use selenium's webdriver, but If I try to change it to the seleniumwire's webdriver, it gave me this proxy error.

Can you kindly tell me how to configure proxy.

'--proxy-server=<container_ip_running_sw>:8087')

for example container_ip of seleniumhub? or which container?

Thank you!