Unable to connect Lighthouse to a remote browser

[brunoklein]

Issue: unable to connect Lighthouse to a remote browser.
Description:

I'm trying to connect Lighthouse to a remote chromium or chrome.
While Playwright and Puppeteer can successfully connect to it, Lighthouse is returning a connection error:

I've also tried different lighthouse hostnames and ports:

hostname: `ws://my-moon-server-ip:4444/playwright/chromium`;
Hostname: `ws://my-moon-server-ip:4444/devtools/chrome-87-0-f81f52b1-231c-4ea9-9bdf-e927d1ea0769`;
port: 4444

I had similar error, eg:

Lighthouse is connecting only on local browser, eg:

const browser = await playwright.chromium.launch({ args: ['--remote-debugging-port=9222'], });

I think I'm missing some step.
How do I properly connect Lighthouse to a remote browser?

Thanks in advance.

Check the code

const lighthouse = require('lighthouse');
const puppeteer = require('puppeteer');
const reportGenerator = require('lighthouse/lighthouse-core/report/report-generator');
const playwright = require('playwright');
const fs = require('fs');
const driver = require('../../lib/utils/driver');

const run = async () => {
    try {
        const testUrl = 'http://google.com';
        const MOON_HOST = 'my-moon-server-ip';
        const MOON_PORT = 4444;
        const playwrightUrl = `ws://${MOON_HOST}:${MOON_PORT}/playwright/chromium`;
        const chromeSessionID = await driver.getSession();
        const devtoolsUrl = `ws://${MOON_HOST}:${MOON_PORT}/devtools/${chromeSessionID}`;

        let options = {
            // hostname: devtoolsUrl,
            // hostname: playwrightUrl,
            // port: 4444,
            logLevel: 'info',
            disableDeviceEmulation: true,
            chromeFlags: [
                '--no-sandbox',
                '--headless',
                '--disable-dev-shm-usage',
                '--disable-mobile-emulation',
                '--ignore-certificate-errors',
            ],
        };

        // REMOTE playwright eg.
        const browserPlaywright = await playwright.chromium.connect({
            wsEndpoint: playwrightUrl,
        });

        // REMOTE puppeteer eg.
        const browserPuppeteer = await puppeteer.connect({
            browserWSEndpoint: devtoolsUrl,
        });

        // LOCAL
        const browser = await playwright.chromium.launch({
            args: ['--remote-debugging-port=9222'],
        });

        const page = await browser.newPage();
        await page.goto(testUrl);
        await page.screenshot({ path: 'output/screenshot.png' });

        // Run Lighthouse
        const { lhr } = await lighthouse(testUrl, options);
        await browser.close();
        const report = reportGenerator.generateReport(lhr, 'json');
        fs.writeFileSync('./output/lighthouse-report.json', report);

    } catch (err) {
        console.log(err);
    }
};

run();

[patrickhulce]

Have you already tried the below?

let options = {
  hostname: MOON_HOST,
  port: MOON_PORT,
  // ...
}

I can't imagine any other variation working successfully, so I'd start there :)

[brendankenny]

It's a little hard to tell from the code you have here, but if Chrome is being launched with --remote-debugging-port=9222, the lighthouse options.port also needs to be 9222.

[brendankenny]

Ah whoops, I missed that the local connection was working but the remote one wasn't. What @patrickhulce said :)

[brunoklein]

Have you already tried the below?

let options = {
  hostname: MOON_HOST,
  port: MOON_PORT,
  // ...
}

I can't imagine any other variation working successfully, so I'd start there :)

Using

hostname: MOON_HOST,
port: MOON_PORT,

returns it:

The same if I access the MOON_HOST:MOON_PORT on my web browser.

[patrickhulce]

Ah, so Moon isn't actually a remote browser. The short answer is you can't do this yet without writing a custom browser connection class that looks like https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/gather/connections/cri.js#L83 (or you implementing some proxy layer to translate Moon). Lighthouse is written to communicate with Chrome and not a generic browser or WebDriver API.

#11313 will make this possible to just use your puppeteer page, but for now it's fairly custom.

[dvelasquez]

Hey @patrickhulce I have a question regarding remote browser usage.

I made this very small proof of concept using Lighthouse and browserless.io Chrome container and is working.

It's as easier as this:

const puppeteer = require('puppeteer-core')
const lighthouse = require('lighthouse');

(async () => {
  try {
    const urlList = ['https://panor.am/places/', 'https://panor.am/', 'https://d13z.dev', 'https://subito.it/']

    for (const url of urlList) {
      const browser = await puppeteer.connect({
        browserWSEndpoint: 'ws://localhost:3000'
      });
      console.log(url)
      const {lhr} = await lighthouse(url, {
        port: (new URL(browser.wsEndpoint())).port,
        output: 'json',
        logLevel: 'silent',
      });

      console.log(`Lighthouse scores: ${Object.values(lhr.categories).map(c => c.score).join(', ')}`);

      await browser.close();
      await new Promise((resolve) => setTimeout(() => resolve(), 3000))
    }
  } catch (error) {
    console.error(error)
  }

})()

Now I'm wondering about the benchmarkIndex. From what I understand, this is calculated by the computeBenchmarkIndex function located in lighthouse/lighthouse-core/lib/page-functions.js.

My question is:

• Is this function executed in the browser or in the NodeJS application?

If this function is benchmarking the NodeJS application and this is different from where the browser is, we could have a misalignment in the benchmarks, because we are not benchmarking what is important - the browser - but instead the application.

[patrickhulce]

Is this function executed in the browser or in the NodeJS application?

It's being executed in the browser wherever the page is being loaded.

lighthouse/lighthouse-core/gather/driver.js

Lines 193 to 199 in 5c32e64

	async getBenchmarkIndex() {
	const status = {msg: 'Benchmarking machine', id: 'lh:gather:getBenchmarkIndex'};
	log.time(status);
	const indexVal = await this.evaluateAsync(`(${pageFunctions.computeBenchmarkIndexString})()`);
	log.timeEnd(status);
	return indexVal;
	}

[dvelasquez]

Great! Thanks @patrickhulce , this means that the benchmarkIndex will be whatever the browser have.

But what if I run that function manually? Would then be the benchmarkIndex of the NodeJS app instead?