Support browser specific protocols (chrome extensions)

[kuceb]

Support cy.visit() with protocols other than http/https such as:

• chrome://
• chrome-extension://
• resource://

Most notably, chrome-extension:// will allow users to test the UI of a chrome extension.

[JulianG]

There's also Add support for testing chrome extensions but it's closed.
I've added a comment because I believe the issue is still happening.

[kuceb]

@JulianG that issue was for testing content scripts, which is currently possible. But yeah this is the other half of Chrome extensions

[pureooze]

Would love to see support for this added!

[timrogers]

I’d be very keen to see this - you can already make a fair bit of headway with extensions, but this would be the icing on the cake.

[vborshchov]

I would love to see this ability. It would be so great.

[kuceb]

Curious, which pages are you most wanting to test? background.html? popup.html?
And you need access to the chrome.* apis correct?
I'm trying to figure out the scope of work here, because embedding the page in an iframe while maintaining access to chrome.* privileged apis might be tricky

[Kocal]

@bkucera For my own use case, I would be able to run Cypress on my extension's popup in order to see if:

• it actually renders
• some dynamic elements are present
• some links are present too
• internal tabs navigation (vue-router) works

I tried cy.visit('chrome-extension://.../popup.html') but it doesn't work due to invalid protocol.

Also, background.html is literally never used because it's a auto-generated page when you use a single background.js (probably ~99% of the time), but it would be nice for running Cypress on options.html page. 👍

Having access to chrome* API would be awesome (e.g.: to check if a notification has been triggered, sync, ...), or maybe we can manually stub chrome* methods at the moment?

Thanks you for taking time on this. 🙂

[JulianG]

@bkucera Exactly. My chrome extension uses chrome.windows.* and chrome.tabs.* and in order to display in a list and manipulate well... windows and tabs.
I'm currently in the process of writing a fake (as in not a mock nor stub) chrome api so I can test in Cypress, but I would very much prefer to test the real thing.

(In my case it's neither background.html nor popup.html. My background.js uses chrome api to open a window containing index.html.)

[pureooze]

In my case I want to test via cy.visit('chrome-extension://.../popup.html') or if possible by using the keyboard hotkeys to trigger the extension. In my case the hotkey causes the background.js to run a script that attaches the HTML for the extension to the DOM. My extension is used to do tab switching so the user could switch to any tab they have open. I also use the history and sessions APIs.

One thing I had hoped to do was call cy.visit a couple times to get cypress to load some browsing history and then run my extension to test the calls to the APIs but I would be willing to settle for stubbing their responses in cypress.

[kuceb]

Thanks everyone for the input. I think the use case of @Kocal can currently be done by stubbing chrome.* apis and running a simple http server in the folder with manifest.json. Then you can just cy.visit http://localhost:8080/popup.html and this should be very similar to visiting chrome-extension://...
I have done this before and you can change the viewport to be something more similar to popup size

[Kocal]

My popup is now successfuly tested, thanks you all! 😎

Some tips for people who wants to spec their extension's views and mocking chrome* API:

• Build your extension (e.g.: output in dist folder)
• Run a local server pointing on your dist folder (serve is pretty nice)
• Update your Cypress configuration:

{
  "baseUrl": "http://localhost:5000"
}

• Into your spec:

describe('App', () => {
  before(() => {

    // Load your popup
    cy.visit('/popup/popup.html', {

      // If you need to stub `chrome*` API, you should do it there:
      onBeforeLoad(win) {
        win.chrome = win.chrome || {};
        win.chrome.runtime = {
          sendMessage(message, cb) {
            // ...
            cb(some_data);
          },
        };
      },
    });
  });
});

[JulianG]

Have you stubbed chrome.* in the end?
I started making a fake implementation of chrome.windows.* and chrome tabs.* which is what I use. Do you know if such thing already exists?

[Kocal]

Well, I only use chrome.runtime.sendMessage inside my popup view, I didn't take time for other methods 😕

EDIT: some times ago I used jest-webextension-mock for mocking chrome.* in Jest context. Maybe it can helps you

[pureooze]

I have used sinon-chrome before, it works well for unit/IT testing. Not sure how it compares to jest-webextension-mock.

[Moejoe90]

For me, the use case would not be testing an extension but testing our app with an extension I want to be able to click some buttons on the extension etc while navigating through our app

[callmenick]

I can see both use cases being strong.

Say you're developing an extension that has some UI flows in the popup, it gets pretty annoying to manually test those each time by clicking the extension icon! Regressions are so hard to notice because of the nature of the popup. With Cypress interface, I'd spot the regressions by just running the test command, and notice exactly where it happens.

On the other side, you're obviously developing an extension to enhance a browser -- and websites as a result -- so running an "example site" inside Cypress with the extension loaded could be useful to notice things like what the extension might inject, etc. like what @Moejoe90 is suggesting.

@bkucera is this being actively worked on? Is there anything the community can do to support?

[kuceb]

@callmenick I'm working on testing a chrome extension that injects html into a third party site, but no Test Runner features are being developed yet.

Most likely I'll end up using something from here to allow the extension to detect content in the AUT (application under test), which cypress puts in an iframe

[callmenick]

Cool, thanks for the update. I'm personally interested in the first point, supporting extension protocols so I can visit chrome-extension://id/index.html and run tests from there. Is there any roadmap for that?

[Kocal]

@callmenick you can check #1965 (comment) I guess

[callmenick]

Yeah that's what we're doing now as well @Kocal. Thanks for that!

[kuceb]

Hey everyone, I was able to set up a good workflow (see however below*) for testing chrome extensions that inject HTML onto third party sites.

For example, to test an extension that injects HTML inside of Gmail, there were a few things I had to do:

1. Export the gmail webpages that need testing using this chrome extension: Save Page WE
   (I saved them to cypress/plugins/gmail/, one of my pages is called gmail_home.html)
2. Host the exported webpages with a express server in plugins/index.js, and it needs to have a self-signed SSL cert: this requires launching cypress with sudo due to needing port 443 (google's port)

plugins/index.js code:

const path = require('path');

module.exports = (on) => {
  on('task', {
    setUrlPath (url) {
      console.log(url)
      filepath = url
      return null
    }
  })
}

let filepath = 'gmail/gmail_home.html'
const express = require('express')

const PORT = 443

const app = express()
const selfSignedHttps = require('self-signed-https')

app.get('/*', (req, res) => {
  return res.sendFile(filepath, { root: __dirname })  
})

selfSignedHttps(app).listen(PORT, () => console.log(`Server running on port ${PORT}`))

...

3. add to cypress.json:

"hosts": {
    "mail.google.com": "127.0.0.1",
}

4. add "all_frames":true to extension's manifest.json

 "content_scripts": [
    {
      "matches": [
        "http://mail.google.com/*",
        "https://mail.google.com/*"
      ],
      "js": [
        ...
      ],
      "css": [
        ...
      ],
      "all_frames": true
    }

4. Finally, load the extension into Cypress by modifying the plugins/index.js file as shown here: https://docs.cypress.io/api/plugins/browser-launch-api.html#Usage

👍

However:

Cypress cy.route stubbing will not affect traffic originating in a chrome extension, so there is more work to do before that can be used.

This will be fixed by Full Network stubbing, and is on our roadmap: #687

[whymarrh]

Similar to @callmenick above, I'm curious about whether or not Cypress can support simply visit a HTML page from a Chrome extension. If an extension has an HTML page listed in its web_accessible_resources it can be loaded in an iframe without too much trouble. That manifest entry, coupled with the --load-extension=$folder flag could allow a very basic UI test of a Chrome extension. Is this something Cypress could support?

[nidasoyab]

My popup is now successfuly tested, thanks you all! 😎

Some tips for people who wants to spec their extension's views and mocking chrome* API:

• Build your extension (e.g.: output in dist folder)
• Run a local server pointing on your dist folder (serve is pretty nice)
• Update your Cypress configuration:

{
  "baseUrl": "http://localhost:5000"
}

• Into your spec:

describe('App', () => {
  before(() => {

    // Load your popup
    cy.visit('/popup/popup.html', {

      // If you need to stub `chrome*` API, you should do it there:
      onBeforeLoad(win) {
        win.chrome = win.chrome || {};
        win.chrome.runtime = {
          sendMessage(message, cb) {
            // ...
            cb(some_data);
          },
        };
      },
    });
  });
});

can you share the repository name of this

[erfanatp]

My popup is now successfuly tested, thanks you all! 😎

Some tips for people who wants to spec their extension's views and mocking chrome* API:

• Build your extension (e.g.: output in dist folder)
• Run a local server pointing on your dist folder (serve is pretty nice)
• Update your Cypress configuration:

{
  "baseUrl": "http://localhost:5000"
}

• Into your spec:

describe('App', () => {
  before(() => {

    // Load your popup
    cy.visit('/popup/popup.html', {

      // If you need to stub `chrome*` API, you should do it there:
      onBeforeLoad(win) {
        win.chrome = win.chrome || {};
        win.chrome.runtime = {
          sendMessage(message, cb) {
            // ...
            cb(some_data);
          },
        };
      },
    });
  });
});

can you share the repository name of this

Serving doesn't work. The browser says "local" and some specific Chrome extension APIs are undefined...

[juandavidkincaid]

For those who are still looking for alternatives or ideas... Try binding puppeteer on a CDP session from the plugins scope, it will enable you to access the chrome-extension:// protocol, something similar to what synpress does, a little bit of work, but completely resolves the issue, at least for now.

Something like this...

const { data: debuggerInfo } = await axios.get(
  `http://localhost:${this.config.debugPort}/json/version`
)

this.browser = await puppeteer.connect({
  browserWSEndpoint: debuggerInfo.webSocketDebuggerUrl,
  ignoreHTTPSErrors: true,
  defaultViewport: null,
})

Then you would access extension pages from puppeteer instead of cypress, do clicks and stuff, but still using and testing with cypress.

[shaun-wild]

Is there any news on this issue?

Really would like to be able to test my extension.

[salimdriai]

My popup is now successfuly tested, thanks you all! 😎

Some tips for people who wants to spec their extension's views and mocking chrome* API:

• Build your extension (e.g.: output in dist folder)
• Run a local server pointing on your dist folder (serve is pretty nice)
• Update your Cypress configuration:

{
  "baseUrl": "http://localhost:5000"
}

• Into your spec:

describe('App', () => {
  before(() => {

    // Load your popup
    cy.visit('/popup/popup.html', {

      // If you need to stub `chrome*` API, you should do it there:
      onBeforeLoad(win) {
        win.chrome = win.chrome || {};
        win.chrome.runtime = {
          sendMessage(message, cb) {
            // ...
            cb(some_data);
          },
        };
      },
    });
  });
});

can someone please explain how to build the extension and point server to it's dist folder

[salimdriai]

For those who are still looking for alternatives or ideas... Try binding puppeteer on a CDP session from the plugins scope, it will enable you to access the chrome-extension:// protocol, something similar to what synpress does, a little bit of work, but completely resolves the issue, at least for now.

Something like this...

const { data: debuggerInfo } = await axios.get(
  `http://localhost:${this.config.debugPort}/json/version`
)

this.browser = await puppeteer.connect({
  browserWSEndpoint: debuggerInfo.webSocketDebuggerUrl,
  ignoreHTTPSErrors: true,
  defaultViewport: null,
})

Then you would access extension pages from puppeteer instead of cypress, do clicks and stuff, but still using and testing with cypress.

is there any guide to implement this ?

[shaun-wild]

I've switched over to Nightwatch for now.

[Depetrol]

same question as @Iskandar47 , any specific guidelines? The introduction is kinda vague and cannot replicate

[juandavidkincaid]

Sorry for the late response @Iskandar47 and @Depetrol... The snippet shared corresponds to an issue related to CDP sessions in synpress... The actual implementation is better understood in synpress code at the moment I don't have enough time to do a demo but is basically the same... The solution uses cypress plugins and puppeteer, cypress reveals a WS socket for debugging purposes and the solution then connects that WS endpoint to puppeteer to control the browser in puppeteer side...

[Depetrol]

Our team have successfully loaded the browser extension page in cypress:

The reason why chrome-extension:// is blocked: cypress runs in-browser and is blocked by chrome security policies

Solution: use puppeteer to control the browser behavior externally with CDP (Chrome Devtools Protocol), bypassing the security limitations.

Here's how to do it:
Load the following code in cypress/plugins/index.js

const axios = require('axios');
const puppeteer = require('puppeteer');
const path = require('path');

const buildDir = path.join(__dirname, '../../build');
const extensionURL = // this is fixed for our extension
  'chrome-extension://abcdefdsbfdsbafdbsfbabsffdbsfbds/index.html';

let debuggingPort = null;

async function setBrowser() {
  if (debuggingPort != null) {
    const url = `http://127.0.0.1:${debuggingPort}/json/version`;
    const { data: debuggerInfo } = await axios.get(url);
    const browser = await puppeteer.connect({
      browserWSEndpoint: debuggerInfo.webSocketDebuggerUrl,
      ignoreHTTPSErrors: true,
      defaultViewport: null,
    });
    const openTabs = await browser.targets();
    const globalPage = await openTabs
      .find(p => p._targetInfo.type === 'page')
      .page();
    const iframeElement = await globalPage.$('iframe.aut-iframe');
    const theFrame = await iframeElement.contentFrame();
    await theFrame.goto(extensionURL);
  } else {
    throw new Error('debuggingPort not provided');
  }

  return null;
}

module.exports = (on, config) => {
  on(
    'before:browser:launch',
    async (browser = { headless: true }, launchOptions) => {
      if (browser.name !== 'electron' || browser.name === 'chromium') {
        // Here you will need to persist the port to your plugins, whatever they may be
        const RDP = launchOptions.args.find(
          arg => arg.slice(0, 23) === '--remote-debugging-port',
        );
        debuggingPort = RDP.split('=')[1];

        launchOptions.extensions.push(buildDir);
      }

      return launchOptions;
    },
  );

  on('task', {
    setBrowser,
  });
};

run the following code before each test

describe('My First Test', () => {
  it('Visits a chrome-extension', () => {
    cy.task('setBrowser')
      .get('.btn-primary')
      .click()
      .children()
      .then(e => console.info(e));
  });
});

[Tofel]

@Depetrol were you able to run these tests in the CI? For me everything works on local (in a fully dockerised setup), but when running in Github Actions it seems that the extension never gets loaded (or at least Puppeteer doesn't find the second tab with the extension within 60 seconds; it's hard to say what's really happening, because Cypress` video/screenshot only includes browser's viewport).

[escapedcat]

@Depetrol cool! This looks like what other people try to do with Playwright as well. Here's an example on how to dynamically get the extensionId: https://github.com/xcv58/Tab-Manager-v2/blob/master/packages/integration_test/util.ts#L51-L55

I recently tried Playwright/Puppeteer and Cypress/Puppeteer and both was working ok. Biggest issue I've noticed for both approaches was that neither could provide the screenshots/videos on fail or or a nice trace of commands. I beleive this happens because Puppeteer is taking over. Not sure though.

Did you get i.e. screenshots on fail or the commands-trace with your Cypress approach? Is that's what the debgging-port is about?

[will-beta]

@Depetrol were you able to run these tests in the CI? For me everything works on local (in a fully dockerised setup), but when running in Github Actions it seems that the extension never gets loaded (or at least Puppeteer doesn't find the second tab with the extension within 60 seconds; it's hard to say what's really happening, because Cypress` video/screenshot only includes browser's viewport).

@Tofel Yes. Try to install a new version of Chrome.

[will-beta]

@Depetrol cool! This looks like what other people try to do with Playwright as well. Here's an example on how to dynamically get the extensionId: https://github.com/xcv58/Tab-Manager-v2/blob/master/packages/integration_test/util.ts#L51-L55

I recently tried Playwright/Puppeteer and Cypress/Puppeteer and both was working ok. Biggest issue I've noticed for both approaches was that neither could provide the screenshots/videos on fail or or a nice trace of commands. I beleive this happens because Puppeteer is taking over. Not sure though.

Did you get i.e. screenshots on fail or the commands-trace with your Cypress approach? Is that's what the debgging-port is about?

@escapedcat Yes. Cypress will automatically capture screenshots when a failure happens during cypress run.

[escapedcat]

@escapedcat Yes. Cypress will automatically capture screenshots when a failure happens during cypress run.

Yeah, that's the default, right? When I tested it using puppeteer (to test an extension) it didn't. It also didn't show a nice command path in the left panel like I'm used to.

Might need to try again.

[Tofel]

No it's not ;-) Cypress knows nothing about Puppeteer, which is used to communicate with MetaMask, so it will never take the screenshot of the extension (regardless whether it's the separate tab it's running in or the notification).

[Tofel]

@Depetrol were you able to run these tests in the CI? For me everything works on local (in a fully dockerised setup), but when running in Github Actions it seems that the extension never gets loaded (or at least Puppeteer doesn't find the second tab with the extension within 60 seconds; it's hard to say what's really happening, because Cypress` video/screenshot only includes browser's viewport).

@Tofel Yes. Try to install a new version of Chrome.

I wished it was that easy, Chrome 101 has memory leaks, which makes it unusable :/

[escapedcat]

No it's not ;-) Cypress knows nothing about Puppeteer, which is used to communicate with MetaMask, so it will never take the screenshot of the extension (regardless whether it's the separate tab it's running in or the notification).

@Tofel right, I think that's what i meant. If you use Cypress only per default Cypress will do all these things but once you handle everything with Puppeteer Cypress doesn't get these infos anymore and won't create i.e. screenshots on fail.

[NorseGaud]

Wow, this is an old thread... I have a need to "visit" a chrome-extension:// URL and load the extension in the current window. I don't need anything else and can run all of my tests once it's loaded. Nothing more, nothing less.

[NorseGaud]

The more I dig in the code, it looks like this may be a change to https://www.npmjs.com/package/@cypress/request which seems to do the request and throw the error.

Furthermore, it seems like the maintainer wants people moving away from it request/request#3142

The best thing for these new modules is for request to slowly fade away, eventually becoming just another memory of that legacy stack. Taking the position request has now and leveraging it for a bigger share of the next generation of developers would be a disservice to those developers as it would drive them away from better modules that don’t have the burden of request’s history.

[NorseGaud]

Our team have successfully loaded the browser extension page in cypress:

The reason why chrome-extension:// is blocked: cypress runs in-browser and is blocked by chrome security policies

Solution: use puppeteer to control the browser behavior externally with CDP (Chrome Devtools Protocol), bypassing the security limitations.

Here's how to do it: Load the following code in cypress/plugins/index.js

const axios = require('axios');
const puppeteer = require('puppeteer');

. . .

This doesn't work. Even with the tweaks to get it to work with the latest puppeteer,await theFrame.goto never completes and all you see is the following with an eventual timeout

[NanoRoss]

Did you manage to make it work?