[Selenium 4] [Grid] Se:Options CDP Endpoint Points to Node

[dylanlive]

🐛 Bug Report

When making a POST /session request to a Selenium 4 Grid Hub, the returned CDP Endpoint points to the node.

Example:

{
    "status": 0,
    "sessionId": "c8adfc6384648299d726d9c63d4e7a15",
    "value": {
        "se:options": {
            "cdp": "http://<node_ip>:5555/session/c8adfc6384648299d726d9c63d4e7a15/se/cdp"
         },
    }
}

Is it intended to not prefer proxying these requests through the Hub (4444)? For WebdriverIO I'm hoping to utilize the URL returned from this response.

For security purposes, I would prefer to lock my Nodes down to only be able to communicate with the Hub, and not have the clients making individual requests to the node.

From my tests back in Alpha, cdp websocket requests to both hub (4444) and node (5555) both were working. So this is more of a decision on which should be advertised.

To Reproduce

1. Launch a Selenium 4 Hub/Node
2. Make a POST /session request

Expected behavior

Response to be:

{
    "status": 0,
    "sessionId": "c8adfc6384648299d726d9c63d4e7a15",
    "value": {
        "se:options": {
            "cdp": "http://<hub_ip>:4444/session/c8adfc6384648299d726d9c63d4e7a15/se/cdp"
         },
    }
}

Test script or set of commands reproducing this issue

curl -d ' {"desiredCapabilities": { "browserName": "chrome" }}' -H 'Content-Type: application/json; charset=utf-8' http://localhost:4444/session

Environment

OS: 10
Browser: Chrome
Browser version: 88.0.4324.150
Browser Driver version: 88.0.4324.96
Language Bindings version: n/a
Selenium Grid version (if applicable): 4.0.0-beta-1

[dylanlive]

Spoke with Diego about this briefly on Slack. There's some more research needed as to the reason why it returns the node in the current implementation.

Did a quick look through the code. Areas I found of interest:

• selenium/java/server/src/org/openqa/selenium/grid/node/config/DriverServiceSessionFactory.java

  Line 178 in e874b53

  return new PersistentCapabilities(caps).setCapability("se:options", ImmutableMap.of("cdp", uri));

• selenium/java/client/src/org/openqa/selenium/chromium/ChromiumDevToolsLocator.java

  Line 46 in e874b53

  public static Optional<URI> getReportedUri(String capabilityKey, Capabilities caps) {

Need to look closer to where it gets the Node IP. Is this being reported directly from the browser driver?

[dylanlive]

This also looks to have consequences with the docker images. Running docker run --rm -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome:4.0.0-beta-1-20210215 and creating a session returns the Docker Internal IP in the response.

[diemol]

You can now start the Node with the flag --grid-url http://192.168.1.7:4444, and then it will be used to rewrite the se:cdp url returned in the capabilities.

For the provided example, it will look like this:

{
  "value": {
    "sessionId": "8c377dcd84c18e465010ce008e29930b",
    "capabilities": {
      "acceptInsecureCerts": false,
      "browserName": "chrome",
      "browserVersion": "89.0.4389.82",
      "chrome": {
        "chromedriverVersion": "89.0.4389.23 (61b08ee2c50024bab004e48d2b1b083cdbdac579-refs/branch-heads/4389@{#294})",
        "userDataDir": "/var/folders/6n/_9hsq46s1gjf3phb6q_fhzph0000gn/T/.com.google.Chrome.eziQ2Q"
      },
      "goog:chromeOptions": {
        "debuggerAddress": "localhost:49335"
      },
      "networkConnectionEnabled": false,
      "pageLoadStrategy": "normal",
      "platformName": "mac os x",
      "proxy": {
      },
      "se:cdp": "http://192.168.1.7:4444/session/8c377dcd84c18e465010ce008e29930b/se/cdp",
      "setWindowRect": true,
      "strictFileInteractability": false,
      "timeouts": {
        "implicit": 0,
        "pageLoad": 300000,
        "script": 30000
      },
      "unhandledPromptBehavior": "dismiss and notify",
      "webauthn:extension:largeBlob": true,
      "webauthn:virtualAuthenticators": true
    }
  }
}

There will be a docker-selenium release soon where one can provide the SE_NODE_GRID_URL env var for the same purpose.

This will be available in beta-2, which should be released during this week.