Gutenberg: Trigger E2E suite for v9.9.x upgrade #49599

WunderBart · 2021-02-02T13:02:37Z

Tracking issue: #49534.

Includes fix for failing tests where inserted block focus is stolen by the restore focus hook. See WordPress/gutenberg#28932 for more details.

ToDo:

A fix for the focus issue mentioned above has been already merged and will be included in the v10.0 release. This PR might need to be reverted when we upgrade to that version.

What about failing specs?

See the summary here: #49599 (comment)

matticbot · 2021-02-02T13:02:38Z

Test live: https://calypso.live/?branch=e2e/gutenberg-upgrade-v9.9

matticbot · 2021-02-02T13:16:54Z

This PR does not affect the size of JS and CSS bundles shipped to the user's browser.

Generated by performance advisor bot at iscalypsofastyet.com.

sirreal · 2021-02-05T10:36:04Z

Rebased and running with 9.9.0 stable on edge.

WunderBart · 2021-02-08T17:46:28Z

Rebased and running with 9.9.1 stable on edge.

fullofcaffeine · 2021-02-11T17:56:14Z

Rebased and running with 9.9.2 stable on edge.

test/e2e/specs/wp-calypso-gutenberg-post-editor-spec.js

WunderBart · 2021-02-12T10:56:06Z

test/e2e/lib/gutenberg/gutenberg-editor-component.js

+			initsWithChildFocus ? 'has-child-selected' : 'is-selected'
+		}[aria-label*='${ ariaLabel }']`;
+
+		let insertedBlockSelector = By.css( insertedBlockSelectorStr );


One of the things I wanted to refactor at some point - the By interface has "locator strategies" that return locators, not selectors. A selector in this case is actually the string passed to the locator strategy (css() in this case). We use the selector naming for locators everywhere, so we should keep doing that to stay consistent, IMO. Just thought I'd mention the "locators" for future reference :)

https://www.selenium.dev/documentation/en/getting_started_with_webdriver/locating_elements/

WunderBart · 2021-02-12T11:02:10Z

Because of this PR that just got merged (planned for v10.1), this might end up not being a workaround at all:

The first thing is that if you insert a block using the inserter, it's not "focused" anymore, which allows the inserter to stay open. From a screen reader perspective, this is correct. Moving the focus was the thing forcing us to actually close the inserter.

We need to keep changes from that PR in mind as they can cause more failed specs in our E2E GB suites.

edit

Looks like we have a fix already! :D WordPress/gutenberg#28962

…by the main inserter Workaround for WordPress/gutenberg#28932. To be reverted once we get a fix from GB core.

WunderBart · 2021-02-12T13:31:39Z

Summary of failing mobile specs:

✅ Currently, there are no failing mobile specs that run on Gutenberg edge sites. See details below.

Mailosaur-related (ignore, see p1612456065018600-slack-C1A1EKDGQ for context):

[WPCOM] Launch (mobile) @signup @parallel Launch a free site "before all" hook: Create a site as a new user for "Can launch a site"
[WPCOM] Launch (mobile) @signup @parallel Launch when having multiple sites "before all" hook: Create 2 free sites as a new user for "Can start launch flow and abandon"
[WPCOM] Launch (mobile) @signup @parallel Launch a free site "before all" hook: Create a site as a new user for "Can launch a site"
[WPCOM] Launch (mobile) @signup @parallel Launch when having multiple sites "before all" hook: Create 2 free sites as a new user for "Can start launch flow and abandon"

Non-Gutenberg-edge-related:

[WPCOM] Themes: All sites (mobile) Activate a theme @parallel when a theme more button is clicked when Activate is clicked Successful activation dialog should show the successful activation dialog
- Doesn't seem to be flaky, failing on master as well.
[WPCOM] Sign Up (mobile, en) Sign up for a non-blog site on a premium paid plan through main flow using a coupon @signup @visdiff Can Remove Coupon
- Flaky, passed locally.
[WPCOM] Plans: (mobile) Viewing a specific plan with coupon: @parallel @jetpack Remove any existing coupon
- Flaky, passed locally.

Gutenberg-related:

[WPCOM] Calypso Gutenberg Editor: Posts (mobile) Insert embeds: @parallel Can publish and view content
- Flaky, passed locally.

Summary of failing desktop specs:

✅ Currently, there are no failing desktop specs that run on Gutenberg edge sites. See details below.

Mailosaur-related (ignore, see p1612456065018600-slack-C1A1EKDGQ for context):

[WPCOM] Launch (desktop) @signup @parallel Launch a free site "before all" hook: Create a site as a new user for "Can launch a site"
[WPCOM] Launch (desktop) @signup @parallel Launch a free site "before all" hook: Create a site as a new user for "Can launch a site"
[WPCOM] Launch (desktop) @signup @parallel Launch when having multiple sites "before all" hook: Create 2 free sites as a new user for "Can start launch flow and abandon"
[WPCOM] Launch (desktop) @signup @parallel Launch when having multiple sites "before all" hook: Create 2 free sites as a new user for "Can start launch flow and abandon"
[WPCOM] Launch (desktop) @signup @parallel Launch a free site "before all" hook: Create a site as a new user for "Can launch a site"
[WPCOM] Launch (desktop) @signup @parallel Launch a free site "before all" hook: Create a site as a new user for "Can launch a site"
[WPCOM] Launch (desktop) @signup @parallel Launch when having multiple sites "before all" hook: Create 2 free sites as a new user for "Can start launch flow and abandon"
[WPCOM] Launch (desktop) @signup @parallel Launch when having multiple sites "before all" hook: Create 2 free sites as a new user for "Can start launch flow and abandon"

Non-Gutenberg-edge-related:

[WPCOM] Themes: All sites (desktop) Preview a theme @parallel when a theme more button is clicked when "Try & Customize" is clicked click try and customize popover
- Doesn't seem to be flaky, failing on master as well.
[WPCOM] Themes: All sites (desktop) Activate a theme @parallel when a theme more button is clicked when Activate is clicked Successful activation dialog should show the successful activation dialog
- Doesn't seem to be flaky, failing on master as well.
[WPCOM] Sign Up (desktop, en) Sign up for a non-blog site on a premium paid plan through main flow using a coupon @signup @visdiff Can Remove Coupon
- Flaky, passed locally.
[WPCOM] Plans: (desktop) Viewing a specific plan with coupon: @parallel @jetpack Remove any existing coupon
- Flaky, passed locally.

Overall, I don't see any specs that are related to Gutenberg v9.9. @fullofcaffeine's fix is working nicely, so I think we can ship this one and move on with the upgrade. cc @griffbrad

This reverts commit 30f1c62.

* Revert "Gutenberg: Trigger E2E suite for v9.9.x upgrade (#49599)" This reverts commit 30f1c62. * Improve the main options menu selector It fails in the mobile viewport because it matches the block context menu, which is returned instead. * Close the welcome modal dialog before trying to select a page template in the mobile screen size * Use clickIfPresent to close the welcome dialog, as it might not show in all scenarios Co-authored-by: Marcelo Serpa <[email protected]>

WunderBart added [Type] Task DO NOT MERGE [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot labels Feb 2, 2021

WunderBart mentioned this pull request Feb 2, 2021

Gutenberg: 9.9 upgrade #49534

Closed

46 tasks

WunderBart force-pushed the e2e/gutenberg-upgrade-v9.9 branch from 84ead44 to f320ac6 Compare February 4, 2021 10:11

sirreal force-pushed the e2e/gutenberg-upgrade-v9.9 branch from f320ac6 to 336ccf8 Compare February 5, 2021 10:35

WunderBart force-pushed the e2e/gutenberg-upgrade-v9.9 branch from 336ccf8 to 7662e0d Compare February 8, 2021 17:39

WunderBart force-pushed the e2e/gutenberg-upgrade-v9.9 branch from 7662e0d to 357bf2a Compare February 11, 2021 13:07

Trigger E2E tests

fd2e54e

fullofcaffeine force-pushed the e2e/gutenberg-upgrade-v9.9 branch from 357bf2a to fd2e54e Compare February 11, 2021 17:56

fullofcaffeine reviewed Feb 11, 2021

View reviewed changes

test/e2e/specs/wp-calypso-gutenberg-post-editor-spec.js Outdated Show resolved Hide resolved

fullofcaffeine force-pushed the e2e/gutenberg-upgrade-v9.9 branch 2 times, most recently from d7ad0c9 to 7fab6d4 Compare February 11, 2021 22:28

fullofcaffeine changed the title ~~Gutenberg: Trigger E2E suite for v9.9 upgrade~~ Gutenberg: Trigger E2E suite for v9.9.x upgrade Feb 11, 2021

WunderBart commented Feb 12, 2021

View reviewed changes

fullofcaffeine added 5 commits February 12, 2021 12:07

Try to find elements that did not receive focus after being inserted …

be21399

…by the main inserter Workaround for WordPress/gutenberg#28932. To be reverted once we get a fix from GB core.

Fix typos in comment

c949c1f

Foce-select the image block so that the contextual settings open for it

f094e13

More comment typo fixes

bc1ac68

Also click the image in the gb page editor spec

d8269e4

WunderBart force-pushed the e2e/gutenberg-upgrade-v9.9 branch from 751d8e5 to d8269e4 Compare February 12, 2021 11:07

WunderBart removed the DO NOT MERGE label Feb 12, 2021

WunderBart marked this pull request as ready for review February 12, 2021 13:45

WunderBart requested a review from fullofcaffeine February 12, 2021 13:48

matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Feb 12, 2021

fullofcaffeine merged commit 30f1c62 into trunk Feb 12, 2021

fullofcaffeine deleted the e2e/gutenberg-upgrade-v9.9 branch February 12, 2021 16:08

matticbot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Feb 12, 2021

WunderBart mentioned this pull request Feb 15, 2021

Gutenberg: Update the E2E suite to run correctly against v9.9.3 #50092

Merged

fullofcaffeine pushed a commit that referenced this pull request Feb 15, 2021

Revert "Gutenberg: Trigger E2E suite for v9.9.x upgrade (#49599)"

6aaba61

This reverts commit 30f1c62.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gutenberg: Trigger E2E suite for v9.9.x upgrade #49599

Gutenberg: Trigger E2E suite for v9.9.x upgrade #49599

WunderBart commented Feb 2, 2021 •

edited

Loading

matticbot commented Feb 2, 2021

matticbot commented Feb 2, 2021 •

edited

Loading

sirreal commented Feb 5, 2021

WunderBart commented Feb 8, 2021

fullofcaffeine commented Feb 11, 2021

WunderBart Feb 12, 2021

WunderBart commented Feb 12, 2021 •

edited

Loading

WunderBart commented Feb 12, 2021 •

edited

Loading

Gutenberg: Trigger E2E suite for v9.9.x upgrade #49599

Gutenberg: Trigger E2E suite for v9.9.x upgrade #49599

Conversation

WunderBart commented Feb 2, 2021 • edited Loading

ToDo:

What about failing specs?

matticbot commented Feb 2, 2021

matticbot commented Feb 2, 2021 • edited Loading

sirreal commented Feb 5, 2021

WunderBart commented Feb 8, 2021

fullofcaffeine commented Feb 11, 2021

WunderBart Feb 12, 2021

Choose a reason for hiding this comment

WunderBart commented Feb 12, 2021 • edited Loading

WunderBart commented Feb 12, 2021 • edited Loading

Summary of failing mobile specs:

Mailosaur-related (ignore, see p1612456065018600-slack-C1A1EKDGQ for context):

Non-Gutenberg-edge-related:

Gutenberg-related:

Summary of failing desktop specs:

Mailosaur-related (ignore, see p1612456065018600-slack-C1A1EKDGQ for context):

Non-Gutenberg-edge-related:

WunderBart commented Feb 2, 2021 •

edited

Loading

matticbot commented Feb 2, 2021 •

edited

Loading

WunderBart commented Feb 12, 2021 •

edited

Loading

WunderBart commented Feb 12, 2021 •

edited

Loading