Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add http proxy settings #254

Merged
merged 3 commits into from
Jul 30, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ after a mysteries note was produced on it.
- [Using headless drivers](#using-headless-drivers)
- [Connection to remote webdriver](#connection-to-remote-webdriver)
- [Webdriver in Docker](#webdriver-in-docker)
- [HTTP Proxy](#http-proxy)
- [Devtools: tracking HTTP requests, XHR (Ajax)](#devtools-tracking-http-requests-xhr-ajax)
- [Postmortem: auto-save artifacts in case of exception](#postmortem-auto-save-artifacts-in-case-of-exception)
- [Reading browser's logs](#reading-browsers-logs)
Expand Down Expand Up @@ -656,6 +657,40 @@ for [Firefox](https://hub.docker.com/r/instrumentisto/geckodriver):
docker run --name geckodriver -p 4444:4444 -d instrumentisto/geckodriver
```

## HTTP Proxy

To set proxy settings use environment variables `HTTP_PROXY`/`HTTPS_PROXY` or pass a map of the following type:

``` clojure
{:proxy {:http "some.proxy.com:8080"
igrishaev marked this conversation as resolved.
Show resolved Hide resolved
:ftp "some.proxy.com:8080"
:ssl "some.proxy.com:8080"
:socks {:host "myproxy:1080" :version 5}
:bypass ["http://this.url" "http://that.url"]
:pac-url "localhost:8888"}}

;; example
(chrome {:proxy {:http "some.proxy.com:8080"
igrishaev marked this conversation as resolved.
Show resolved Hide resolved
:ssl "some.proxy.com:8080"}})
```
Note: A :pac-url for a [proxy autoconfiguration file](https://en.wikipedia.org/wiki/Proxy_auto-config#The_PAC_File).
Used with Safari as the other proxy options do not work in that browser.

To fine tune the proxy you can use the original [object](https://www.w3.org/TR/webdriver/#proxy) and pass it to capabilities:

``` clojure
{:capabilities {:proxy {:proxyType "manual"
:proxyAutoconfigUrl "some.proxy.com:8080"
:ftpProxy "some.proxy.com:8080"
:httpProxy "some.proxy.com:8080"
:noProxy ["http://this.url" "http://that.url"]
:sslProxy "some.proxy.com:8080"
:socksProxy "some.proxy.com:1080"
:socksVersion 5}}}

(chrome {:capabilities {:proxy {...}}})
```

## Devtools: tracking HTTP requests, XHR (Ajax)

With recent updates, the library brings a great feature. Now you can trace
Expand Down
23 changes: 23 additions & 0 deletions src/etaoin/api.clj
Original file line number Diff line number Diff line change
Expand Up @@ -2811,6 +2811,14 @@
driver))


(defn proxy-env
[proxy]
(let [http (System/getenv "HTTP_PROXY")
ssl (System/getenv "HTTPS_PROXY")]
(cond-> proxy
http (assoc :http http)
ssl (assoc :ssl ssl))))

(defn run-driver
"Runs a driver process locally.

Expand Down Expand Up @@ -2853,6 +2861,17 @@
rather than local machine. Currently, only FF and Chrome support headless mode.
Phantom.js is headless by its nature.

-- `proxy` is a map of proxy server connection settings.

--- `http` is a string. Defines the proxy host for HTTP traffic.
--- `ssl` is a string. Defines the proxy host for encrypted TLS traffic.
--- `ftp` is a string. Defines the proxy host for FTP traffic.
--- `pac-url` is a string. Defines the URL for a proxy auto-config file.
--- `bypass` is a vector. Lists the address for which the proxy should be bypassed.
--- `socks` is a map.
---- `host` is a string. Defines the proxy host for a SOCKS proxy.
---- `version` Any integer between 0 and 255 inclusive. Defines the SOCKS proxy version.

-- `:args` is a vector of additional command line arguments
to the browser's process.

Expand All @@ -2871,6 +2890,7 @@
args
size
prefs
proxy
profile
headless
log-level
Expand All @@ -2884,6 +2904,7 @@
[with height] size
log-level (or log-level :all)
path-driver (or path-driver (get-in defaults [type :path]))
proxy (proxy-env proxy)

_ (swap! driver drv/set-browser-log-level log-level)
_ (swap! driver drv/set-path path-driver)
Expand All @@ -2905,6 +2926,8 @@
(swap! driver drv/set-headless))
_ (when args
(swap! driver drv/set-options-args args))
_ (when proxy
(swap! driver drv/set-proxy proxy))
_ (when profile
(swap! driver drv/set-profile profile))
_ (when path-browser
Expand Down
24 changes: 24 additions & 0 deletions src/etaoin/driver.clj
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,30 @@
[driver]
true)

;;
;; HTTP proxy
;;

(defn proxy->w3c
[proxy]
(let [{:keys [http ssl ftp socks pac-url bypass]} proxy]
(cond-> nil
(or ssl http
ftp socks) (assoc :proxyType "manual")
pac-url (assoc :proxyType "pac"
:proxyAutoconfigUrl pac-url)
http (assoc :httpProxy http)
ssl (assoc :sslProxy ssl)
ftp (assoc :ftpProxy ftp)
socks (assoc :socksProxy (:host socks)
:socksVersion (or (:version socks) 5))
bypass (assoc :noProxy bypass))))

(defn set-proxy
[driver proxy]
(let [proxy-w3c (proxy->w3c proxy)]
(set-capabilities driver {:proxy proxy-w3c})))

;;
;; Custom preferences
;;
Expand Down