Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try stabilizing customize widgets e2e tests #32023

Merged

Conversation

talldan
Copy link
Contributor

@talldan talldan commented May 20, 2021

Description

Attempts to solve the intermittent failure in the customize widgets tests observed here - https://github.com/WordPress/gutenberg/runs/2620274056.

The test attempts to insert a paragraph and a heading, and then use the trailing appender to add a Search block.

But the test failure snapshots show that the test is sometimes inserting two paragraphs instead.

My guess is that the Paragraph is incorrectly inserted due to the animation the block library panel has in the customizer. The test is attempting to click the Heading block while the animation is happening, but clicks the Paragraph block that's next to it.

There isn't a trailing appender in this situation, because the appender doesn't appear after paragraphs, so the test fails.

This PR adds a timeout to wait for the animation to finish. The animation is 0.18 seconds, so waiting 300ms should be plenty.

(edit: animations could be disabled entirely in e2e tests when this issue is solved - #32024)

@talldan talldan requested review from ajitbohra, nerrad and ntwb as code owners May 20, 2021 03:40
@talldan talldan self-assigned this May 20, 2021
@talldan talldan added [Feature] Widgets Customizer Ability to add and edit blocks in Customize → Widgets. [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. labels May 20, 2021
@talldan talldan requested review from kevin940726 and ellatrix May 20, 2021 03:44
@github-actions
Copy link

Size Change: 0 B

Total Size: 1.62 MB

ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.12 kB 0 B
build/annotations/index.js 2.93 kB 0 B
build/api-fetch/index.js 2.42 kB 0 B
build/autop/index.js 2.28 kB 0 B
build/blob/index.js 673 B 0 B
build/block-directory/index.js 6.61 kB 0 B
build/block-directory/style-rtl.css 989 B 0 B
build/block-directory/style.css 990 B 0 B
build/block-editor/index.js 119 kB 0 B
build/block-editor/style-rtl.css 12.8 kB 0 B
build/block-editor/style.css 12.8 kB 0 B
build/block-library/blocks/archives/editor-rtl.css 61 B 0 B
build/block-library/blocks/archives/editor.css 60 B 0 B
build/block-library/blocks/audio/editor-rtl.css 58 B 0 B
build/block-library/blocks/audio/editor.css 58 B 0 B
build/block-library/blocks/audio/style-rtl.css 112 B 0 B
build/block-library/blocks/audio/style.css 112 B 0 B
build/block-library/blocks/block/editor-rtl.css 161 B 0 B
build/block-library/blocks/block/editor.css 161 B 0 B
build/block-library/blocks/button/editor-rtl.css 475 B 0 B
build/block-library/blocks/button/editor.css 474 B 0 B
build/block-library/blocks/button/style-rtl.css 601 B 0 B
build/block-library/blocks/button/style.css 600 B 0 B
build/block-library/blocks/buttons/editor-rtl.css 315 B 0 B
build/block-library/blocks/buttons/editor.css 315 B 0 B
build/block-library/blocks/buttons/style-rtl.css 375 B 0 B
build/block-library/blocks/buttons/style.css 375 B 0 B
build/block-library/blocks/calendar/style-rtl.css 208 B 0 B
build/block-library/blocks/calendar/style.css 208 B 0 B
build/block-library/blocks/categories/editor-rtl.css 84 B 0 B
build/block-library/blocks/categories/editor.css 83 B 0 B
build/block-library/blocks/categories/style-rtl.css 79 B 0 B
build/block-library/blocks/categories/style.css 79 B 0 B
build/block-library/blocks/code/style-rtl.css 90 B 0 B
build/block-library/blocks/code/style.css 90 B 0 B
build/block-library/blocks/columns/editor-rtl.css 190 B 0 B
build/block-library/blocks/columns/editor.css 190 B 0 B
build/block-library/blocks/columns/style-rtl.css 422 B 0 B
build/block-library/blocks/columns/style.css 422 B 0 B
build/block-library/blocks/cover/editor-rtl.css 643 B 0 B
build/block-library/blocks/cover/editor.css 645 B 0 B
build/block-library/blocks/cover/style-rtl.css 1.22 kB 0 B
build/block-library/blocks/cover/style.css 1.22 kB 0 B
build/block-library/blocks/embed/editor-rtl.css 486 B 0 B
build/block-library/blocks/embed/editor.css 486 B 0 B
build/block-library/blocks/embed/style-rtl.css 401 B 0 B
build/block-library/blocks/embed/style.css 400 B 0 B
build/block-library/blocks/file/editor-rtl.css 301 B 0 B
build/block-library/blocks/file/editor.css 300 B 0 B
build/block-library/blocks/file/frontend.js 771 B 0 B
build/block-library/blocks/file/style-rtl.css 255 B 0 B
build/block-library/blocks/file/style.css 255 B 0 B
build/block-library/blocks/freeform/editor-rtl.css 2.45 kB 0 B
build/block-library/blocks/freeform/editor.css 2.45 kB 0 B
build/block-library/blocks/gallery/editor-rtl.css 704 B 0 B
build/block-library/blocks/gallery/editor.css 705 B 0 B
build/block-library/blocks/gallery/style-rtl.css 1.06 kB 0 B
build/block-library/blocks/gallery/style.css 1.05 kB 0 B
build/block-library/blocks/group/editor-rtl.css 160 B 0 B
build/block-library/blocks/group/editor.css 160 B 0 B
build/block-library/blocks/group/style-rtl.css 57 B 0 B
build/block-library/blocks/group/style.css 57 B 0 B
build/block-library/blocks/heading/editor-rtl.css 129 B 0 B
build/block-library/blocks/heading/editor.css 129 B 0 B
build/block-library/blocks/heading/style-rtl.css 76 B 0 B
build/block-library/blocks/heading/style.css 76 B 0 B
build/block-library/blocks/home-link/style-rtl.css 259 B 0 B
build/block-library/blocks/home-link/style.css 259 B 0 B
build/block-library/blocks/html/editor-rtl.css 281 B 0 B
build/block-library/blocks/html/editor.css 281 B 0 B
build/block-library/blocks/image/editor-rtl.css 717 B 0 B
build/block-library/blocks/image/editor.css 716 B 0 B
build/block-library/blocks/image/style-rtl.css 481 B 0 B
build/block-library/blocks/image/style.css 485 B 0 B
build/block-library/blocks/latest-comments/style-rtl.css 281 B 0 B
build/block-library/blocks/latest-comments/style.css 282 B 0 B
build/block-library/blocks/latest-posts/editor-rtl.css 137 B 0 B
build/block-library/blocks/latest-posts/editor.css 137 B 0 B
build/block-library/blocks/latest-posts/style-rtl.css 523 B 0 B
build/block-library/blocks/latest-posts/style.css 522 B 0 B
build/block-library/blocks/legacy-widget/editor-rtl.css 557 B 0 B
build/block-library/blocks/legacy-widget/editor.css 557 B 0 B
build/block-library/blocks/list/style-rtl.css 63 B 0 B
build/block-library/blocks/list/style.css 63 B 0 B
build/block-library/blocks/media-text/editor-rtl.css 176 B 0 B
build/block-library/blocks/media-text/editor.css 176 B 0 B
build/block-library/blocks/media-text/style-rtl.css 492 B 0 B
build/block-library/blocks/media-text/style.css 489 B 0 B
build/block-library/blocks/more/editor-rtl.css 434 B 0 B
build/block-library/blocks/more/editor.css 434 B 0 B
build/block-library/blocks/navigation-link/editor-rtl.css 633 B 0 B
build/block-library/blocks/navigation-link/editor.css 634 B 0 B
build/block-library/blocks/navigation-link/style-rtl.css 94 B 0 B
build/block-library/blocks/navigation-link/style.css 94 B 0 B
build/block-library/blocks/navigation/editor-rtl.css 1.54 kB 0 B
build/block-library/blocks/navigation/editor.css 1.54 kB 0 B
build/block-library/blocks/navigation/style-rtl.css 1.78 kB 0 B
build/block-library/blocks/navigation/style.css 1.78 kB 0 B
build/block-library/blocks/nextpage/editor-rtl.css 395 B 0 B
build/block-library/blocks/nextpage/editor.css 395 B 0 B
build/block-library/blocks/page-list/editor-rtl.css 310 B 0 B
build/block-library/blocks/page-list/editor.css 311 B 0 B
build/block-library/blocks/page-list/style-rtl.css 233 B 0 B
build/block-library/blocks/page-list/style.css 233 B 0 B
build/block-library/blocks/paragraph/editor-rtl.css 157 B 0 B
build/block-library/blocks/paragraph/editor.css 157 B 0 B
build/block-library/blocks/paragraph/style-rtl.css 247 B 0 B
build/block-library/blocks/paragraph/style.css 248 B 0 B
build/block-library/blocks/post-author/editor-rtl.css 209 B 0 B
build/block-library/blocks/post-author/editor.css 209 B 0 B
build/block-library/blocks/post-author/style-rtl.css 183 B 0 B
build/block-library/blocks/post-author/style.css 184 B 0 B
build/block-library/blocks/post-comments-form/style-rtl.css 140 B 0 B
build/block-library/blocks/post-comments-form/style.css 140 B 0 B
build/block-library/blocks/post-comments/style-rtl.css 360 B 0 B
build/block-library/blocks/post-comments/style.css 359 B 0 B
build/block-library/blocks/post-content/editor-rtl.css 139 B 0 B
build/block-library/blocks/post-content/editor.css 139 B 0 B
build/block-library/blocks/post-excerpt/editor-rtl.css 73 B 0 B
build/block-library/blocks/post-excerpt/editor.css 73 B 0 B
build/block-library/blocks/post-excerpt/style-rtl.css 69 B 0 B
build/block-library/blocks/post-excerpt/style.css 69 B 0 B
build/block-library/blocks/post-featured-image/editor-rtl.css 338 B 0 B
build/block-library/blocks/post-featured-image/editor.css 338 B 0 B
build/block-library/blocks/post-featured-image/style-rtl.css 119 B 0 B
build/block-library/blocks/post-featured-image/style.css 119 B 0 B
build/block-library/blocks/post-title/style-rtl.css 60 B 0 B
build/block-library/blocks/post-title/style.css 60 B 0 B
build/block-library/blocks/preformatted/style-rtl.css 103 B 0 B
build/block-library/blocks/preformatted/style.css 103 B 0 B
build/block-library/blocks/pullquote/editor-rtl.css 183 B 0 B
build/block-library/blocks/pullquote/editor.css 183 B 0 B
build/block-library/blocks/pullquote/style-rtl.css 318 B 0 B
build/block-library/blocks/pullquote/style.css 318 B 0 B
build/block-library/blocks/query-loop/editor-rtl.css 98 B 0 B
build/block-library/blocks/query-loop/editor.css 97 B 0 B
build/block-library/blocks/query-loop/style-rtl.css 315 B 0 B
build/block-library/blocks/query-loop/style.css 317 B 0 B
build/block-library/blocks/query-pagination-numbers/editor-rtl.css 122 B 0 B
build/block-library/blocks/query-pagination-numbers/editor.css 121 B 0 B
build/block-library/blocks/query-pagination/editor-rtl.css 270 B 0 B
build/block-library/blocks/query-pagination/editor.css 262 B 0 B
build/block-library/blocks/query-pagination/style-rtl.css 168 B 0 B
build/block-library/blocks/query-pagination/style.css 168 B 0 B
build/block-library/blocks/query-title/editor-rtl.css 86 B 0 B
build/block-library/blocks/query-title/editor.css 86 B 0 B
build/block-library/blocks/query/editor-rtl.css 131 B 0 B
build/block-library/blocks/query/editor.css 132 B 0 B
build/block-library/blocks/quote/style-rtl.css 169 B 0 B
build/block-library/blocks/quote/style.css 169 B 0 B
build/block-library/blocks/rss/editor-rtl.css 201 B 0 B
build/block-library/blocks/rss/editor.css 202 B 0 B
build/block-library/blocks/rss/style-rtl.css 290 B 0 B
build/block-library/blocks/rss/style.css 290 B 0 B
build/block-library/blocks/search/editor-rtl.css 189 B 0 B
build/block-library/blocks/search/editor.css 189 B 0 B
build/block-library/blocks/search/style-rtl.css 359 B 0 B
build/block-library/blocks/search/style.css 362 B 0 B
build/block-library/blocks/separator/editor-rtl.css 99 B 0 B
build/block-library/blocks/separator/editor.css 99 B 0 B
build/block-library/blocks/separator/style-rtl.css 251 B 0 B
build/block-library/blocks/separator/style.css 251 B 0 B
build/block-library/blocks/shortcode/editor-rtl.css 512 B 0 B
build/block-library/blocks/shortcode/editor.css 512 B 0 B
build/block-library/blocks/site-logo/editor-rtl.css 440 B 0 B
build/block-library/blocks/site-logo/editor.css 441 B 0 B
build/block-library/blocks/site-logo/style-rtl.css 154 B 0 B
build/block-library/blocks/site-logo/style.css 154 B 0 B
build/block-library/blocks/social-link/editor-rtl.css 164 B 0 B
build/block-library/blocks/social-link/editor.css 165 B 0 B
build/block-library/blocks/social-links/editor-rtl.css 800 B 0 B
build/block-library/blocks/social-links/editor.css 799 B 0 B
build/block-library/blocks/social-links/style-rtl.css 1.32 kB 0 B
build/block-library/blocks/social-links/style.css 1.33 kB 0 B
build/block-library/blocks/spacer/editor-rtl.css 308 B 0 B
build/block-library/blocks/spacer/editor.css 308 B 0 B
build/block-library/blocks/spacer/style-rtl.css 48 B 0 B
build/block-library/blocks/spacer/style.css 48 B 0 B
build/block-library/blocks/table/editor-rtl.css 478 B 0 B
build/block-library/blocks/table/editor.css 478 B 0 B
build/block-library/blocks/table/style-rtl.css 485 B 0 B
build/block-library/blocks/table/style.css 485 B 0 B
build/block-library/blocks/tag-cloud/editor-rtl.css 118 B 0 B
build/block-library/blocks/tag-cloud/editor.css 118 B 0 B
build/block-library/blocks/tag-cloud/style-rtl.css 94 B 0 B
build/block-library/blocks/tag-cloud/style.css 94 B 0 B
build/block-library/blocks/template-part/editor-rtl.css 551 B 0 B
build/block-library/blocks/template-part/editor.css 550 B 0 B
build/block-library/blocks/term-description/editor-rtl.css 90 B 0 B
build/block-library/blocks/term-description/editor.css 90 B 0 B
build/block-library/blocks/text-columns/editor-rtl.css 95 B 0 B
build/block-library/blocks/text-columns/editor.css 95 B 0 B
build/block-library/blocks/text-columns/style-rtl.css 166 B 0 B
build/block-library/blocks/text-columns/style.css 166 B 0 B
build/block-library/blocks/verse/style-rtl.css 87 B 0 B
build/block-library/blocks/verse/style.css 87 B 0 B
build/block-library/blocks/video/editor-rtl.css 569 B 0 B
build/block-library/blocks/video/editor.css 570 B 0 B
build/block-library/blocks/video/style-rtl.css 169 B 0 B
build/block-library/blocks/video/style.css 169 B 0 B
build/block-library/common-rtl.css 1.26 kB 0 B
build/block-library/common.css 1.26 kB 0 B
build/block-library/editor-rtl.css 9.93 kB 0 B
build/block-library/editor.css 9.92 kB 0 B
build/block-library/index.js 147 kB 0 B
build/block-library/reset-rtl.css 506 B 0 B
build/block-library/reset.css 507 B 0 B
build/block-library/style-rtl.css 10.2 kB 0 B
build/block-library/style.css 10.3 kB 0 B
build/block-library/theme-rtl.css 692 B 0 B
build/block-library/theme.css 693 B 0 B
build/block-serialization-default-parser/index.js 1.29 kB 0 B
build/block-serialization-spec-parser/index.js 3.06 kB 0 B
build/blocks/index.js 47.1 kB 0 B
build/components/index.js 188 kB 0 B
build/components/style-rtl.css 16.2 kB 0 B
build/components/style.css 16.2 kB 0 B
build/compose/index.js 9.95 kB 0 B
build/core-data/index.js 12.1 kB 0 B
build/customize-widgets/index.js 43.2 kB 0 B
build/customize-widgets/style-rtl.css 1.38 kB 0 B
build/customize-widgets/style.css 1.38 kB 0 B
build/data-controls/index.js 829 B 0 B
build/data/index.js 7.23 kB 0 B
build/date/index.js 31.8 kB 0 B
build/deprecated/index.js 739 B 0 B
build/dom-ready/index.js 577 B 0 B
build/dom/index.js 4.62 kB 0 B
build/edit-navigation/index.js 13.6 kB 0 B
build/edit-navigation/style-rtl.css 2.82 kB 0 B
build/edit-navigation/style.css 2.82 kB 0 B
build/edit-post/classic-rtl.css 454 B 0 B
build/edit-post/classic.css 454 B 0 B
build/edit-post/index.js 335 kB 0 B
build/edit-post/style-rtl.css 6.8 kB 0 B
build/edit-post/style.css 6.79 kB 0 B
build/edit-site/index.js 25.6 kB 0 B
build/edit-site/style-rtl.css 4.76 kB 0 B
build/edit-site/style.css 4.75 kB 0 B
build/edit-widgets/index.js 292 kB 0 B
build/edit-widgets/style-rtl.css 3.46 kB 0 B
build/edit-widgets/style.css 3.47 kB 0 B
build/editor/index.js 38.4 kB 0 B
build/editor/style-rtl.css 3.92 kB 0 B
build/editor/style.css 3.91 kB 0 B
build/element/index.js 3.44 kB 0 B
build/escape-html/index.js 739 B 0 B
build/format-library/index.js 5.66 kB 0 B
build/format-library/style-rtl.css 637 B 0 B
build/format-library/style.css 639 B 0 B
build/hooks/index.js 1.76 kB 0 B
build/html-entities/index.js 627 B 0 B
build/i18n/index.js 3.73 kB 0 B
build/is-shallow-equal/index.js 710 B 0 B
build/keyboard-shortcuts/index.js 1.65 kB 0 B
build/keycodes/index.js 1.43 kB 0 B
build/list-reusable-blocks/index.js 2.06 kB 0 B
build/list-reusable-blocks/style-rtl.css 629 B 0 B
build/list-reusable-blocks/style.css 628 B 0 B
build/media-utils/index.js 3.08 kB 0 B
build/navigation/index.js 2.85 kB 0 B
build/notices/index.js 1.07 kB 0 B
build/nux/index.js 2.31 kB 0 B
build/nux/style-rtl.css 718 B 0 B
build/nux/style.css 716 B 0 B
build/plugins/index.js 1.99 kB 0 B
build/primitives/index.js 1.03 kB 0 B
build/priority-queue/index.js 791 B 0 B
build/react-i18n/index.js 923 B 0 B
build/redux-routine/index.js 2.82 kB 0 B
build/reusable-blocks/index.js 2.54 kB 0 B
build/reusable-blocks/style-rtl.css 225 B 0 B
build/reusable-blocks/style.css 225 B 0 B
build/rich-text/index.js 10.7 kB 0 B
build/server-side-render/index.js 1.64 kB 0 B
build/shortcode/index.js 1.68 kB 0 B
build/token-list/index.js 846 B 0 B
build/url/index.js 1.95 kB 0 B
build/viewport/index.js 1.28 kB 0 B
build/warning/index.js 1.13 kB 0 B
build/widgets/index.js 1.66 kB 0 B
build/wordcount/index.js 1.24 kB 0 B

compressed-size-action

Copy link
Member

@kevin940726 kevin940726 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's give it a try!

@talldan talldan merged commit 3f45f6a into trunk May 20, 2021
@talldan talldan deleted the add/wait-for-widget-block-library-panel-animation-in-e2e-test branch May 20, 2021 04:20
@talldan talldan added the [Type] Task Issues or PRs that have been broken down into an individual action to take label May 20, 2021
@github-actions github-actions bot added this to the Gutenberg 10.7 milestone May 20, 2021
//
// eslint-disable-next-line no-restricted-syntax
await page.waitForTimeout( 300 );

const blockOption = await find( {
role: 'option',
name: blockName,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we use normal selectors here? And then we can just wait for the selector?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I guess that wouldn't work with the animation anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

find does wait for the element, but yeah, the problem wasn't about that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Feature] Widgets Customizer Ability to add and edit blocks in Customize → Widgets. [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Task Issues or PRs that have been broken down into an individual action to take
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants