Add Playwright e2e test suite ready for incremental migration and visual regression testing #7671
Conversation
Test the buildOption 1. Jetpack Beta
Option 2. Jurassic Ninja - available for logged-in A12s🚀 Launch a JN site with this branch 🚀 ℹ️ Install this Tampermonkey script to get more options. Build info:
Note: the build is updated when a new commit is pushed to this PR. |
|
Size Change: -3.73 kB (0%) Total Size: 1.25 MB
ℹ️ View Unchanged
|
This comment was marked as resolved.
This comment was marked as resolved.
Jinksi
changed the title
Take a partial screenshot focusing only on the WCPay page element to avoid flakiness caused by general notices and banners.
To review required changes and refactors to also get rid of @woocommerce/e2e-utils
|
Thanks for taking a look @reykjalin 🙌
It shouldn't be impacted by this PR, how does this script go on I'd suggest running |
There was a problem hiding this comment.
This is all looking really good! Got this working in Rancher desktop too, so you don't even need Docker 🙌
npm run test:e2e-pw-ui only showed the auth tests though, not sure why that's happening?
@reykjalin This might happen when restarting the container (re-running E.g. You can fix this by visited |
Hmm, that doesn't seem to work 🤔 . |
|
Hey @reykjalin, that's probably because Playwright UI only shows the first project by default, which is the setup part in our case. |
There was a problem hiding this comment.
Hey @reykjalin, that's probably because Playwright UI only shows the first project by default, which is the setup part in our case.
Oh, that's exactly it! I didn't even think to click that, and even if I had I wouldn't have thought to click "chromium". Is there a way we can change the name of the project?
Also, it seems that basic.spec.ts and auth.setup.ts serve a very similar purpose. Or at least they're basically the same tests. It's probably enough to keep only one of those? Or somehow merge them?
I think we could rename it here, AFAIK that's the actual name, the browser comes from the |
|
Reading the docs, PW projects lean towards being used to designate testing for multiple browsers, hence the default But I've attempted a 
I think I tried it merged with |
|
@achyuthajoy would you still like to re-review this PR? |
There was a problem hiding this comment.
Thanks for working @Jinksi. Tested the changes and works well.
LGTM 🚢
|
Thanks! I'll aim to merge this ASAP ( |
|
Failing |
Resolves #8177
Changes proposed in this Pull Request
This PR lays the foundation for Playwright e2e and visual regression testing, ready for incremental migration of existing Puppeteer e2e tests.
Migrating our e2e tests to use Playwright instead of Puppeteer will provide us benefits including, but not limited to:
To aid in migration, this PR follows the WooCommerce Puppeteer to Playwright migration guide which adds a Playwright test suite alongside the existing Puppeteer test suite.
TODO
tests/e2e-pw/README.mdtests/e2e-pw/README.mdfor getting started)Notes/inclusions:
e2e-pwdirectory).developortrunkand outputs the Playwright HTML report as a GH action artefact (see this workflow run)Testing instructions
npm installnpm run test:e2e-setupnpm run test:e2e-upnpm run test:e2e-pwheadless run from within a linux docker container.npm run test:e2e-pw-uiruns tests in interactive UI mode from within a linux docker container – recommended for authoring tests and re-running failed tests.npm run test:e2e-pw keywordruns tests only with a specific keyword in the file name, e.g.disputeorcheckout.npm run test:e2e-pw --update-snapshotsupdates snapshots.E2E Playwright Testspassesplaywright-report.zipindex.htmllocally in your browser to view the test reportnpm run changelogto add a changelog file, choosepatchto leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.Post merge