Skip to content
This repository has been archived by the owner on Feb 23, 2024. It is now read-only.

WIP: Run E2E test suite on WordPress 6.4 #11451

Closed
wants to merge 11 commits into from
Closed

Conversation

gigitux
Copy link
Contributor

@gigitux gigitux commented Oct 25, 2023

What

This PR fixes the failing E2E tests when using WordPress 6.4

Why

Testing Instructions

Please consider any edge cases this change may have, and also other areas of the product this may impact.

  1. Confirm all tests are passing in this PR
  • Do not include in the Testing Notes
  • Should be tested by the development team exclusively

Screenshots or screencast

Before After

WooCommerce Visibility

Required:

  • WooCommerce Core
  • Feature plugin
  • Experimental
  • N/A

Checklist

Required:

  • This PR has either a [type] label or a [skip-changelog] label.
  • This PR is assigned to a milestone.

Conditional:

  • This PR has a changelog description (if [skip-changelog] label is not present).
  • This PR adds/removes a feature flag & I've updated this doc.
  • This PR adds/removes an experimental interfaces, and I've updated this doc.
  • This PR has been accessibility tested.
  • This PR has had any necessary documentation added/updated.

Changelog

Add suggested changelog entry here.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 25, 2023

The release ZIP for this PR is accessible via:

https://wcblocks.wpcomstaging.com/wp-content/uploads/woocommerce-gutenberg-products-block-11451.zip

Script Dependencies Report

There is no changed script dependency between this branch and trunk.

This comment was automatically generated by the ./github/compare-assets action.

TypeScript Errors Report

  • Files with errors: 575
  • Total errors: 2483

🎉 🎉 This PR does not introduce new TS errors.

comments-aggregator

@github-actions
Copy link
Contributor

github-actions bot commented Oct 25, 2023

Size Change: +411 B (0%)

Total Size: 1.54 MB

Filename Size Change
build/blocks-components.js 8.72 kB +25 B (0%)
build/product-collection.js 13.9 kB +373 B (+3%)
build/product-query.js 11.7 kB +13 B (0%)
ℹ️ View Unchanged
Filename Size
build/3810-frontend.js 18.3 kB
build/4124-frontend.js 23.9 kB
build/8280-frontend.js 8.48 kB
build/active-filters-frontend.js 6.69 kB
build/active-filters-rtl.css 1.68 kB
build/active-filters-wrapper-frontend.js 6.9 kB
build/active-filters-wrapper-rtl.css 1.53 kB
build/active-filters-wrapper.css 1.53 kB
build/active-filters.css 1.68 kB
build/active-filters.js 6.06 kB
build/add-to-cart-form-rtl.css 444 B
build/add-to-cart-form.css 444 B
build/all-products-frontend.js 9.68 kB
build/all-products-rtl.css 4.54 kB
build/all-products.css 4.54 kB
build/all-products.js 39.7 kB
build/all-reviews-rtl.css 1.75 kB
build/all-reviews.css 1.75 kB
build/all-reviews.js 7.8 kB
build/attribute-filter-frontend.js 19.9 kB
build/attribute-filter-rtl.css 4.03 kB
build/attribute-filter-wrapper-frontend.js 21.1 kB
build/attribute-filter-wrapper-rtl.css 1.65 kB
build/attribute-filter-wrapper.css 1.65 kB
build/attribute-filter.css 4.01 kB
build/attribute-filter.js 11.3 kB
build/base-components-stock-filter-wrapper~attribute-filter-wrapper~rating-filter-wrapper-style.scss-rtl.css 2.97 kB
build/base-components-stock-filter-wrapper~attribute-filter-wrapper~rating-filter-wrapper-style.scss.css 2.97 kB
build/blocks-checkout.js 33.9 kB
build/breadcrumbs-rtl.css 234 B
build/breadcrumbs.css 234 B
build/breadcrumbs.js 2.13 kB
build/cart-blocks/cart-accepted-payment-methods-frontend.js 1.4 kB
build/cart-blocks/cart-accepted-payment-methods-style.js 153 B
build/cart-blocks/cart-cross-sells-frontend.js 267 B
build/cart-blocks/cart-cross-sells-products-frontend.js 5.56 kB
build/cart-blocks/cart-cross-sells-products-style.js 153 B
build/cart-blocks/cart-cross-sells-style.js 269 B
build/cart-blocks/cart-express-payment-frontend.js 5.37 kB
build/cart-blocks/cart-express-payment-style.js 155 B
build/cart-blocks/cart-items-frontend.js 281 B
build/cart-blocks/cart-items-style.js 240 B
build/cart-blocks/cart-line-items-frontend.js 9.24 kB
build/cart-blocks/cart-line-items-style.js 153 B
build/cart-blocks/cart-order-summary-frontend.js 20.6 kB
build/cart-blocks/cart-order-summary-style.js 339 B
build/cart-blocks/cart-totals-frontend.js 296 B
build/cart-blocks/cart-totals-style.js 253 B
build/cart-blocks/empty-cart-frontend.js 376 B
build/cart-blocks/empty-cart-style.js 375 B
build/cart-blocks/filled-cart-frontend.js 614 B
build/cart-blocks/filled-cart-style.js 332 B
build/cart-blocks/order-summary-coupon-form-frontend.js 20.9 kB
build/cart-blocks/order-summary-coupon-form-style.js 155 B
build/cart-blocks/order-summary-discount-frontend.js 21 kB
build/cart-blocks/order-summary-discount-style.js 155 B
build/cart-blocks/order-summary-fee-frontend.js 288 B
build/cart-blocks/order-summary-fee-style.js 153 B
build/cart-blocks/order-summary-heading-frontend.js 347 B
build/cart-blocks/order-summary-heading-style.js 351 B
build/cart-blocks/order-summary-shipping-frontend.js 20.5 kB
build/cart-blocks/order-summary-shipping-style.js 154 B
build/cart-blocks/order-summary-subtotal-frontend.js 291 B
build/cart-blocks/order-summary-subtotal-style.js 154 B
build/cart-blocks/order-summary-taxes-frontend.js 456 B
build/cart-blocks/order-summary-taxes-style.js 202 B
build/cart-blocks/proceed-to-checkout-frontend.js 7.64 kB
build/cart-blocks/proceed-to-checkout-style.js 1.09 kB
build/cart-frontend.js 29 kB
build/cart-rtl.css 9.24 kB
build/cart.css 9.23 kB
build/cart.js 39.7 kB
build/catalog-sorting-rtl.css 259 B
build/catalog-sorting.css 259 B
build/catalog-sorting.js 1.7 kB
build/checkout-blocks/actions-frontend.js 8.14 kB
build/checkout-blocks/actions-style.js 1.01 kB
build/checkout-blocks/billing-address-frontend.js 9.98 kB
build/checkout-blocks/billing-address-style.js 639 B
build/checkout-blocks/contact-information-frontend.js 1.7 kB
build/checkout-blocks/contact-information-style.js 653 B
build/checkout-blocks/express-payment-frontend.js 5.82 kB
build/checkout-blocks/fields-frontend.js 375 B
build/checkout-blocks/fields-style.js 342 B
build/checkout-blocks/order-note-frontend.js 681 B
build/checkout-blocks/order-summary-cart-items-frontend.js 6.48 kB
build/checkout-blocks/order-summary-cart-items-style.js 153 B
build/checkout-blocks/order-summary-coupon-form-frontend.js 20.9 kB
build/checkout-blocks/order-summary-coupon-form-style.js 155 B
build/checkout-blocks/order-summary-discount-frontend.js 21 kB
build/checkout-blocks/order-summary-discount-style.js 154 B
build/checkout-blocks/order-summary-fee-frontend.js 291 B
build/checkout-blocks/order-summary-fee-style.js 155 B
build/checkout-blocks/order-summary-frontend.js 20.6 kB
build/checkout-blocks/order-summary-shipping-frontend.js 20.6 kB
build/checkout-blocks/order-summary-shipping-style.js 154 B
build/checkout-blocks/order-summary-style.js 341 B
build/checkout-blocks/order-summary-subtotal-frontend.js 289 B
build/checkout-blocks/order-summary-subtotal-style.js 155 B
build/checkout-blocks/order-summary-taxes-frontend.js 455 B
build/checkout-blocks/order-summary-taxes-style.js 201 B
build/checkout-blocks/payment-frontend.js 14.5 kB
build/checkout-blocks/payment-style.js 509 B
build/checkout-blocks/pickup-options-frontend.js 11.2 kB
build/checkout-blocks/pickup-options-style.js 481 B
build/checkout-blocks/shipping-address-frontend.js 9.94 kB
build/checkout-blocks/shipping-address-style.js 517 B
build/checkout-blocks/shipping-method-frontend.js 1.97 kB
build/checkout-blocks/shipping-method-style.js 1.44 kB
build/checkout-blocks/shipping-methods-frontend.js 19 kB
build/checkout-blocks/shipping-methods-style.js 456 B
build/checkout-blocks/terms-frontend.js 1.56 kB
build/checkout-blocks/terms-style.js 1.03 kB
build/checkout-blocks/totals-frontend.js 339 B
build/checkout-blocks/totals-style.js 300 B
build/checkout-frontend.js 30.6 kB
build/checkout-rtl.css 8.43 kB
build/checkout.css 8.42 kB
build/checkout.js 42.8 kB
build/classic-shortcode-rtl.css 242 B
build/classic-shortcode.css 241 B
build/classic-shortcode.js 4.66 kB
build/collection-filters.js 2.74 kB
build/collection-price-filter-frontend.js 591 B
build/collection-price-filter-rtl.css 1.07 kB
build/collection-price-filter.css 1.07 kB
build/collection-price-filter.js 2.15 kB
build/customer-account-rtl.css 410 B
build/customer-account.css 409 B
build/customer-account.js 3.19 kB
build/featured-category-rtl.css 974 B
build/featured-category.css 973 B
build/featured-category.js 13.6 kB
build/featured-product-rtl.css 1.02 kB
build/featured-product.css 1.02 kB
build/featured-product.js 13.8 kB
build/filter-wrapper-frontend.js 14.6 kB
build/filter-wrapper-rtl.css 378 B
build/filter-wrapper.css 378 B
build/filter-wrapper.js 2.38 kB
build/handpicked-products.js 7.33 kB
build/legacy-template-rtl.css 240 B
build/legacy-template.css 240 B
build/legacy-template.js 7.85 kB
build/mini-cart-component-frontend.js 30.8 kB
build/mini-cart-contents-block/cart-button-frontend.js 1.86 kB
build/mini-cart-contents-block/cart-button-style.js 1.23 kB
build/mini-cart-contents-block/checkout-button-frontend.js 1.95 kB
build/mini-cart-contents-block/checkout-button-style.js 1.44 kB
build/mini-cart-contents-block/empty-cart-frontend.js 383 B
build/mini-cart-contents-block/empty-cart-style.js 387 B
build/mini-cart-contents-block/filled-cart-frontend.js 284 B
build/mini-cart-contents-block/filled-cart-style.js 287 B
build/mini-cart-contents-block/footer-frontend.js 3.87 kB
build/mini-cart-contents-block/footer-style.js 1.96 kB
build/mini-cart-contents-block/items-frontend.js 246 B
build/mini-cart-contents-block/items-style.js 250 B
build/mini-cart-contents-block/products-table-frontend.js 8.59 kB
build/mini-cart-contents-block/shopping-button-frontend.js 501 B
build/mini-cart-contents-block/shopping-button-style.js 361 B
build/mini-cart-contents-block/title-frontend.js 2.04 kB
build/mini-cart-contents-block/title-items-counter-frontend.js 1.74 kB
build/mini-cart-contents-block/title-items-counter-style.js 1.2 kB
build/mini-cart-contents-block/title-label-frontend.js 1.68 kB
build/mini-cart-contents-block/title-label-style.js 1.14 kB
build/mini-cart-contents-block/title-style.js 1.38 kB
build/mini-cart-contents-rtl.css 3.22 kB
build/mini-cart-contents.css 3.21 kB
build/mini-cart-contents.js 16 kB
build/mini-cart-frontend.js 2.35 kB
build/mini-cart-rtl.css 2.44 kB
build/mini-cart.css 2.44 kB
build/mini-cart.js 6.09 kB
build/order-confirmation-additional-information-rtl.css 367 B
build/order-confirmation-additional-information.css 367 B
build/order-confirmation-additional-information.js 1.58 kB
build/order-confirmation-billing-address-rtl.css 398 B
build/order-confirmation-billing-address.css 397 B
build/order-confirmation-billing-address.js 1.56 kB
build/order-confirmation-billing-wrapper.js 1.51 kB
build/order-confirmation-downloads-rtl.css 477 B
build/order-confirmation-downloads-wrapper.js 1.58 kB
build/order-confirmation-downloads.css 478 B
build/order-confirmation-downloads.js 1.91 kB
build/order-confirmation-shipping-address-rtl.css 399 B
build/order-confirmation-shipping-address.css 398 B
build/order-confirmation-shipping-address.js 1.56 kB
build/order-confirmation-shipping-wrapper.js 1.52 kB
build/order-confirmation-status-rtl.css 280 B
build/order-confirmation-status.css 280 B
build/order-confirmation-status.js 1.55 kB
build/order-confirmation-summary-rtl.css 460 B
build/order-confirmation-summary.css 460 B
build/order-confirmation-summary.js 1.76 kB
build/order-confirmation-totals-rtl.css 594 B
build/order-confirmation-totals-wrapper.js 1.8 kB
build/order-confirmation-totals.css 593 B
build/order-confirmation-totals.js 2.18 kB
build/packages-style-rtl.css 5.18 kB
build/packages-style.css 5.19 kB
build/page-content-wrapper.js 1.96 kB
build/price-filter-frontend.js 7.94 kB
build/price-filter-rtl.css 2.68 kB
build/price-filter-wrapper-frontend.js 8.11 kB
build/price-filter-wrapper-rtl.css 2.53 kB
build/price-filter-wrapper.css 2.53 kB
build/price-filter.css 2.67 kB
build/price-filter.js 7.53 kB
build/price-format.js 913 B
build/product-add-to-cart-frontend.js 8.12 kB
build/product-add-to-cart-rtl.css 1.37 kB
build/product-add-to-cart.css 1.38 kB
build/product-add-to-cart.js 8.36 kB
build/product-average-rating-frontend.js 1.88 kB
build/product-average-rating.js 1.4 kB
build/product-best-sellers.js 7.07 kB
build/product-button-frontend.js 4.94 kB
build/product-button-interactivity-frontend.js 8.47 kB
build/product-button-rtl.css 1.14 kB
build/product-button.css 1.14 kB
build/product-button.js 4.66 kB
build/product-categories-rtl.css 654 B
build/product-categories.css 654 B
build/product-categories.js 2.6 kB
build/product-category.js 8 kB
build/product-details-rtl.css 397 B
build/product-details.css 394 B
build/product-gallery-frontend.js 837 B
build/product-gallery-large-image-frontend.js 648 B
build/product-gallery-large-image-next-previous.js 4.25 kB
build/product-gallery-large-image.js 2.47 kB
build/product-gallery-pager.js 3.48 kB
build/product-gallery-rtl.css 1.26 kB
build/product-gallery-thumbnails.js 3.9 kB
build/product-gallery.css 1.26 kB
build/product-gallery.js 9.57 kB
build/product-image-frontend.js 2.89 kB
build/product-image-gallery-rtl.css 307 B
build/product-image-gallery.css 306 B
build/product-image-rtl.css 996 B
build/product-image.css 994 B
build/product-image.js 2.57 kB
build/product-new.js 7.33 kB
build/product-on-sale.js 7.32 kB
build/product-price-frontend.js 2.82 kB
build/product-price-rtl.css 644 B
build/product-price.css 643 B
build/product-price.js 2.34 kB
build/product-query-rtl.css 350 B
build/product-query.css 349 B
build/product-rating-counter-frontend.js 2.19 kB
build/product-rating-counter.js 1.7 kB
build/product-rating-frontend.js 2.53 kB
build/product-rating-rtl.css 247 B
build/product-rating-stars-frontend.js 2.43 kB
build/product-rating-stars-rtl.css 899 B
build/product-rating-stars.css 900 B
build/product-rating-stars.js 1.95 kB
build/product-rating.css 246 B
build/product-rating.js 2.04 kB
build/product-results-count-rtl.css 230 B
build/product-results-count.css 230 B
build/product-results-count.js 1.66 kB
build/product-reviews-rtl.css 458 B
build/product-reviews.css 458 B
build/product-sale-badge-frontend.js 2.01 kB
build/product-sale-badge-rtl.css 437 B
build/product-sale-badge.css 437 B
build/product-sale-badge.js 1.52 kB
build/product-search-rtl.css 419 B
build/product-search.css 417 B
build/product-search.js 2.62 kB
build/product-sku-frontend.js 2.02 kB
build/product-sku-rtl.css 240 B
build/product-sku.css 239 B
build/product-sku.js 1.53 kB
build/product-stock-indicator-frontend.js 2.2 kB
build/product-stock-indicator-rtl.css 232 B
build/product-stock-indicator.css 232 B
build/product-stock-indicator.js 1.71 kB
build/product-summary-frontend.js 2.36 kB
build/product-summary-rtl.css 549 B
build/product-summary.css 549 B
build/product-summary.js 1.88 kB
build/product-tag.js 7.52 kB
build/product-template-rtl.css 536 B
build/product-template.css 535 B
build/product-template.js 2.81 kB
build/product-title-frontend.js 2.31 kB
build/product-title-rtl.css 693 B
build/product-title.css 694 B
build/product-title.js 2.05 kB
build/product-top-rated.js 7.6 kB
build/products-by-attribute.js 8.05 kB
build/rating-filter-frontend.js 18.8 kB
build/rating-filter-rtl.css 4.09 kB
build/rating-filter-wrapper-frontend.js 19.7 kB
build/rating-filter-wrapper-rtl.css 1.73 kB
build/rating-filter-wrapper.css 1.73 kB
build/rating-filter.css 4.08 kB
build/rating-filter.js 5.8 kB
build/reviews-by-category-rtl.css 1.75 kB
build/reviews-by-category.css 1.75 kB
build/reviews-by-category.js 11.4 kB
build/reviews-by-product-rtl.css 1.75 kB
build/reviews-by-product.css 1.75 kB
build/reviews-by-product.js 12.7 kB
build/reviews-frontend.js 6.44 kB
build/single-product-rtl.css 378 B
build/single-product.css 378 B
build/single-product.js 11.1 kB
build/stock-filter-frontend.js 19 kB
build/stock-filter-rtl.css 3.88 kB
build/stock-filter-wrapper-frontend.js 20 kB
build/stock-filter-wrapper-rtl.css 1.49 kB
build/stock-filter-wrapper.css 1.49 kB
build/stock-filter.css 3.87 kB
build/stock-filter.js 6.44 kB
build/store-notices.js 1.68 kB
build/wc-blocks-classic-template-revert-button-style-rtl.css 240 B
build/wc-blocks-classic-template-revert-button-style.css 239 B
build/wc-blocks-classic-template-revert-button.js 1.19 kB
build/wc-blocks-data.js 19.7 kB
build/wc-blocks-editor-style-rtl.css 7.07 kB
build/wc-blocks-editor-style.css 7.06 kB
build/wc-blocks-google-analytics.js 1.16 kB
build/wc-blocks-jetpack-woocommerce-analytics.js 750 B
build/wc-blocks-middleware.js 735 B
build/wc-blocks-registry.js 2.75 kB
build/wc-blocks-rtl.css 2.48 kB
build/wc-blocks-shared-context.js 860 B
build/wc-blocks-shared-hocs.js 1.41 kB
build/wc-blocks-vendors.js 61.7 kB
build/wc-blocks.css 2.48 kB
build/wc-blocks.js 2.63 kB
build/wc-interactivity.js 10.7 kB
build/wc-payment-method-bacs.js 405 B
build/wc-payment-method-cheque.js 402 B
build/wc-payment-method-cod.js 508 B
build/wc-payment-method-paypal.js 439 B
build/wc-settings.js 2.4 kB
build/wc-shipping-method-pickup-location.js 29.4 kB

compressed-size-action

@tarhi-saad
Copy link
Contributor

Hi @gigitux! Are we planning to merge this PR when all tests pass? Our work of fixing the failing E2E tests is still in progress. I'll need to jump on another task this week. Please let me know if I should create a separate ticket to unblock this PR!

@gigitux
Copy link
Contributor Author

gigitux commented Nov 1, 2023

Are we planning to merge this PR when all tests pass?

I think so.

Please let me know if I should create a separate ticket to unblock this PR!

If it is necessary, I will ping you! Thanks for working on this PR 🙇

@tarhi-saad
Copy link
Contributor

@gigitux, there is no block for the shop page for some reason! There should be some failing tests because of this. The Playwright Tests / Playwright E2E tests - SideEffects (pull_request) check is canceled. I suspect we have some failing tests depending on the Product Catalog in there.

image

@gigitux
Copy link
Contributor Author

gigitux commented Nov 7, 2023

Hey @tarhi-saad,
Thanks for working on this PR!

It is expected that the shop page doesn't contain any blocks. I checked the commit da705a1 and I noticed that some E2E tests regarding the cart template are still failing.

image

The Playwright Tests / Playwright E2E tests - SideEffects (pull_request) check is canceled.

Good catch about this! I hope that I fixed this wrong behavior with 796582d

@tarhi-saad
Copy link
Contributor

@gigitux, we've created ticket woocommerce/woocommerce#42119 in response to the regression detected with WP 6.4, as discussed on Slack (p1699340273337689-slack-C8X6Q7XQU), which is also behind the failed E2E tests in the current PR. Please let us know if you need this PR merged quickly and what we can do to help move things along.

@gigitux
Copy link
Contributor Author

gigitux commented Nov 8, 2023

@gigitux, we've created ticket woocommerce/woocommerce#42119 in response to the regression detected with WP 6.4, as discussed on Slack (p1699340273337689-slack-C8X6Q7XQU), which is also behind the failed E2E tests in the current PR. Please let us know if you need this PR merged quickly and what we can do to help move things along.

WordPress has been released, so wp-env.json will use WordPress 6.4 by default, for this reason, currently on trunk, E2E tests are failing.

Since you are aware of the issue and you tracked it, I would suggest skipping the failed tests and merge this PR (if other tests don't fail)

@tarhi-saad, does it make sense?

@tarhi-saad tarhi-saad added skip-changelog PRs that you don't want to appear in the changelog. category: tests labels Nov 8, 2023
@gigitux
Copy link
Contributor Author

gigitux commented Nov 8, 2023

I'm closing this PR because we fixed it with #11700.

However, we noticed weird behavior with E2E tests (caching issues?), @tarhi-saad will open an issue about this!

@gigitux gigitux closed this Nov 8, 2023
@tarhi-saad
Copy link
Contributor

Thank you for closing this PR, @gigitux!

However, we noticed weird behavior with E2E tests (caching issues?), @tarhi-saad will open an issue about this!

I was doing some tests while writing the ticket for this issue, and I found out that the wp-env in trunk is now displaying the correct heading for this test:

test( 'Template can be opened in the site editor', async ( {
admin,
page,
editorUtils,
} ) => {
await admin.visitAdminPage( 'site-editor.php' );
await editorUtils.waitForSiteEditorFinishLoading();
await page.getByRole( 'button', { name: /Templates/i } ).click();
await page.getByRole( 'button', { name: /Page: Cart/i } ).click();
await editorUtils.enterEditMode();
await expect(
page
.frameLocator( 'iframe[title="Editor canvas"i]' )
.locator( 'h2:has-text("Cart")' )
.first()
).toBeVisible();
} );
. It doesn't fail if we switch the HTML tag from h2 to h1. I don't have another example to check to write my ticket. I'll double-check by creating a test PR. The same thing we did in #11451, and check if any other failing tests appear.

@tarhi-saad tarhi-saad deleted the add/e2e-tests-6.4 branch November 28, 2023 07:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
metric: flaky e2e test skip-changelog PRs that you don't want to appear in the changelog.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants