Improve E2E checkout tests

[mdmoore]

Fixes #6882

Changes proposed in this Pull Request

This is adding more test coverage for the deferred intent UPE. The split UPE tests were ported over to test with the deferred intent flag enabled instead. The e2e coverage is already in pretty good shape and covers the main checkout flows. I've added a couple more covering the following scenarios:

• Guest cannot save card
• Default payment method can be set on Payment Methods page

The default payment method test has added some flakiness that I haven't been able to pin down even though the test looks fine in the browser.

#5022 (comment) is a great reference for potential tests in the future.

---

• Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
• Covered with tests (or have a good reason not to test in description ☝️)
• Tested on mobile (or does not apply)

Post merge

• Link to testing instructions from release testing doc following these instructions : Add link here / 'QA Testing Not Applicable'
• Add or update critical flows and testing instructions for critical flows, if applicable.
• Add what's changed (description, screenshot, demo videos etc.) to the release announcement post, if applicable.

[botwoo]

Test the build

Option 1. Jetpack Beta

• Install and activate Jetpack Beta.
• Use this build by searching for PR number 7664 or branch name test/deferred-intent in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.

---

Build info:

• Latest commit: 77e751a
• Build time: 2023-12-13 15:16:25 UTC

Note: the build is updated when a new commit is pushed to this PR.

[github-actions]

Size Change: 0 B

Total Size: 1.26 MB

ℹ️ View Unchanged

Filename	Size
release/woocommerce-payments/assets/css/admin.css	1.06 kB
release/woocommerce-payments/assets/css/success.css	158 B
release/woocommerce-payments/dist/blocks-checkout-rtl.css	1.8 kB
release/woocommerce-payments/dist/blocks-checkout.css	1.8 kB
release/woocommerce-payments/dist/blocks-checkout.js	85.5 kB
release/woocommerce-payments/dist/checkout-rtl.css	318 B
release/woocommerce-payments/dist/checkout.css	319 B
release/woocommerce-payments/dist/checkout.js	37.1 kB
release/woocommerce-payments/dist/index-rtl.css	36.8 kB
release/woocommerce-payments/dist/index.css	36.8 kB
release/woocommerce-payments/dist/index.js	287 kB
release/woocommerce-payments/dist/multi-currency-analytics.js	1.05 kB
release/woocommerce-payments/dist/multi-currency-rtl.css	3.4 kB
release/woocommerce-payments/dist/multi-currency-switcher-block.js	60.5 kB
release/woocommerce-payments/dist/multi-currency.css	3.4 kB
release/woocommerce-payments/dist/multi-currency.js	55.7 kB
release/woocommerce-payments/dist/order-rtl.css	676 B
release/woocommerce-payments/dist/order.css	679 B
release/woocommerce-payments/dist/order.js	41.7 kB
release/woocommerce-payments/dist/payment-gateways-rtl.css	1.31 kB
release/woocommerce-payments/dist/payment-gateways.css	1.31 kB
release/woocommerce-payments/dist/payment-gateways.js	39.3 kB
release/woocommerce-payments/dist/payment-request-rtl.css	153 B
release/woocommerce-payments/dist/payment-request.css	153 B
release/woocommerce-payments/dist/payment-request.js	13.6 kB
release/woocommerce-payments/dist/product-details.js	919 B
release/woocommerce-payments/dist/settings-rtl.css	10.3 kB
release/woocommerce-payments/dist/settings.css	10.4 kB
release/woocommerce-payments/dist/settings.js	233 kB
release/woocommerce-payments/dist/subscription-edit-page.js	669 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css	519 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css	519 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js	20.4 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js	710 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css	117 B
release/woocommerce-payments/dist/subscriptions-empty-state.css	117 B
release/woocommerce-payments/dist/subscriptions-empty-state.js	19.5 kB
release/woocommerce-payments/dist/tos-rtl.css	230 B
release/woocommerce-payments/dist/tos.css	231 B
release/woocommerce-payments/dist/tos.js	22 kB
release/woocommerce-payments/dist/woopay-express-button-rtl.css	153 B
release/woocommerce-payments/dist/woopay-express-button.css	153 B
release/woocommerce-payments/dist/woopay-express-button.js	52.6 kB
release/woocommerce-payments/dist/woopay-rtl.css	4.18 kB
release/woocommerce-payments/dist/woopay.css	4.19 kB
release/woocommerce-payments/dist/woopay.js	71.8 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css	622 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js	812 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js	2.43 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js	1.01 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js	522 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js	581 B
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/babel.config.js	160 B
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.css	2.32 kB
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.js	13.8 kB
release/woocommerce-payments/vendor/automattic/jetpack-identity-crisis/build/index.rtl.css	2.32 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css	1.03 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css	291 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css	403 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css	3.6 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css	299 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css	742 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css	572 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css	411 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js	544 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js	9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js	6.8 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js	3.83 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js	544 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js	2.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js	22.1 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js	11.6 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js	1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js	502 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js	355 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js	429 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js	1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js	781 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js	1.1 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js	1.27 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css	392 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js	3.05 kB

_{compressed-size-action}

[timur27]

First round of the review

Currently, I'm noticing that the new test-class-upe-deferred-intent.php is identical to the split-upe-gateway test with the caveat that it sets the different feature flag.

Since we now execute everything (including unit tests) with the deferred intent creation flag on by default, the existing test and the one added in this PR perform the same checks. Having said that, I'd vote for keeping only the e2e tests changes in this PR.

Context on what we want to improve further if it goes to test coverage

We would want to merge test-class-wc-payment-gateway-wcpay.php, test-class-upe-payment-gateway.php and test-class-upe-split-payment-gateway.php into a single test, which will be part of the clean up work and hopefully can be achieved easily. @timur27 will cover this

Another goal is to extend unit test coverage in some places that still require coverage. Those are not that easy to identify, but here's an example on what we'll be possibly adding and I'm taking care of it as part of #7770 and will continue working on it in the next cleanup parts.

Next steps for this PR

• @timur27 to review the e2e tests next week because there is a higher priority now on having this PR merged by EOD Monday.
• keep only e2e tests in this PR

[timur27]

The code is ready for the review. The test was flaky both on this branch and on develop, and I think I fixed this. Out of five consequent runs

1. passed
2. passed
3. passed
4. passed
5. passed

What's left to review my changes

• review the above comments that I left on my commits
• review the code
• re-run GH Actions to triple-check tests are passing
• possibly run tests/e2e/specs/upe-split/shopper/shopper-deferred-intent-creation-upe-enabled.spec.js locally
  • locally this suite is failing because giropay checkbox selector has different ID from the one used in the plugin version run on GitHub. Ideally we need to avoid magic numbers like #inspector-checkbox-control-4 and look for the checkbox by its text. This improvement is out of the scope for this PR, but to run the test locally, change '#inspector-checkbox-control-7', // giropay to '#inspector-checkbox-control-4', // giropay in shopper-deferred-intent-creation-upe-enabled.spec.js. If something is still not working, env clean up and setting up from scratch should help
• approve if all above is green

[mdmoore]

✅ review the above comments that I left on my commits
✅ review the code
✅ re-run GH Actions to triple-check tests are passing
Only compatility tests are failing currently as currently expected. Tests are passing.

✅ possibly run tests/e2e/specs/upe-split/shopper/shopper-deferred-intent-creation-upe-enabled.spec.js locally

locally this suite is failing because giropay checkbox selector has different ID from the one used in the plugin version run on GitHub. Ideally we need to avoid magic numbers like #inspector-checkbox-control-4 and look for the checkbox by its text. This improvement is out of the scope for this PR, but to run the test locally, change '#inspector-checkbox-control-7', // giropay to '#inspector-checkbox-control-4', // giropay in shopper-deferred-intent-creation-upe-enabled.spec.js. If something is still not working, env clean up and setting up from scratch should help

I was getting failures locally with this change. It turns out that #inspector-checkbox-control-7 was correct in my case 🤷‍♂️. With that ID, the everything passed locally.

✅ approve if all above is green

[brettshumaker]

LGTM