Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some basic e2e tests for the widgets screen #28160

Merged
merged 8 commits into from
Jan 14, 2021

Conversation

kevin940726
Copy link
Member

@kevin940726 kevin940726 commented Jan 13, 2021

Description

Close #25784.

An attempt to add some basic e2e tests for the widgets screen following the rules in #27260 but without introducing a new library yet.

The selectors are mostly using aria attributes or human-readable text, to simulate what the actual users see/read. Some exceptions are made to use data attribute where there's no stable a11y queries available.

You will see a lot of TODOs and FIXMEs here 😅. We're planning to fix them in following PRs or open issues.

The top priority for this PR is to make sure the tests pass consistently.

How has this been tested?

npm run test-e2e -- packages/e2e-tests/specs/widgets/adding-widgets.test.js

Types of changes

Bug fix

Checklist:

  • My code is tested.
  • My code follows the WordPress code style.
  • My code follows the accessibility standards.
  • My code has proper inline documentation.
  • I've included developer documentation if appropriate.
  • I've updated all React Native files affected by any refactorings/renamings in this PR.

@kevin940726 kevin940726 added [Type] Enhancement A suggestion for improvement. [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Feature] Widgets Screen The block-based screen that replaced widgets.php. labels Jan 13, 2021
@github-actions
Copy link

github-actions bot commented Jan 13, 2021

Size Change: +188 B (0%)

Total Size: 1.28 MB

Filename Size Change
build/annotations/index.js 3.8 kB -3 B (0%)
build/block-editor/style-rtl.css 11.5 kB +10 B (0%)
build/block-editor/style.css 11.5 kB +11 B (0%)
build/block-library/blocks/cover/editor-rtl.css 524 B +16 B (+3%)
build/block-library/blocks/cover/editor.css 522 B +16 B (+3%)
build/block-library/blocks/cover/style-rtl.css 1.3 kB -22 B (-2%)
build/block-library/blocks/cover/style.css 1.3 kB -21 B (-2%)
build/block-library/editor-rtl.css 8.94 kB +6 B (0%)
build/block-library/editor.css 8.94 kB +6 B (0%)
build/block-library/index.js 142 kB +336 B (0%)
build/block-library/style-rtl.css 8.5 kB -23 B (0%)
build/block-library/style.css 8.5 kB -19 B (0%)
build/core-data/index.js 15.2 kB +23 B (0%)
build/edit-post/style-rtl.css 6.56 kB -81 B (-1%)
build/edit-post/style.css 6.55 kB -78 B (-1%)
build/edit-site/index.js 24.2 kB +190 B (+1%)
build/edit-site/style-rtl.css 4 kB -43 B (-1%)
build/edit-site/style.css 4 kB -40 B (-1%)
build/edit-widgets/style-rtl.css 3.16 kB -58 B (-2%)
build/edit-widgets/style.css 3.16 kB -57 B (-2%)
build/i18n/index.js 3.57 kB -1 B (0%)
build/rich-text/index.js 13.5 kB +20 B (0%)
ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.14 kB 0 B
build/api-fetch/index.js 3.42 kB 0 B
build/autop/index.js 2.83 kB 0 B
build/blob/index.js 665 B 0 B
build/block-directory/index.js 9.03 kB 0 B
build/block-directory/style-rtl.css 1.01 kB 0 B
build/block-directory/style.css 1.01 kB 0 B
build/block-editor/index.js 121 kB 0 B
build/block-library/blocks/archives/editor-rtl.css 196 B 0 B
build/block-library/blocks/archives/editor.css 196 B 0 B
build/block-library/blocks/audio/editor-rtl.css 194 B 0 B
build/block-library/blocks/audio/editor.css 194 B 0 B
build/block-library/blocks/audio/style-rtl.css 225 B 0 B
build/block-library/blocks/audio/style.css 225 B 0 B
build/block-library/blocks/block/editor-rtl.css 283 B 0 B
build/block-library/blocks/block/editor.css 283 B 0 B
build/block-library/blocks/button/editor-rtl.css 576 B 0 B
build/block-library/blocks/button/editor.css 577 B 0 B
build/block-library/blocks/button/style-rtl.css 552 B 0 B
build/block-library/blocks/button/style.css 552 B 0 B
build/block-library/blocks/buttons/editor-rtl.css 345 B 0 B
build/block-library/blocks/buttons/editor.css 346 B 0 B
build/block-library/blocks/buttons/style-rtl.css 419 B 0 B
build/block-library/blocks/buttons/style.css 419 B 0 B
build/block-library/blocks/calendar/style-rtl.css 319 B 0 B
build/block-library/blocks/calendar/style.css 319 B 0 B
build/block-library/blocks/categories/editor-rtl.css 210 B 0 B
build/block-library/blocks/categories/editor.css 209 B 0 B
build/block-library/blocks/categories/style-rtl.css 208 B 0 B
build/block-library/blocks/categories/style.css 208 B 0 B
build/block-library/blocks/code/style-rtl.css 216 B 0 B
build/block-library/blocks/code/style.css 216 B 0 B
build/block-library/blocks/columns/editor-rtl.css 300 B 0 B
build/block-library/blocks/columns/editor.css 299 B 0 B
build/block-library/blocks/columns/style-rtl.css 529 B 0 B
build/block-library/blocks/columns/style.css 528 B 0 B
build/block-library/blocks/embed/editor-rtl.css 594 B 0 B
build/block-library/blocks/embed/editor.css 595 B 0 B
build/block-library/blocks/embed/style-rtl.css 489 B 0 B
build/block-library/blocks/embed/style.css 489 B 0 B
build/block-library/blocks/file/editor-rtl.css 314 B 0 B
build/block-library/blocks/file/editor.css 313 B 0 B
build/block-library/blocks/file/style-rtl.css 352 B 0 B
build/block-library/blocks/file/style.css 352 B 0 B
build/block-library/blocks/freeform/editor-rtl.css 2.55 kB 0 B
build/block-library/blocks/freeform/editor.css 2.55 kB 0 B
build/block-library/blocks/gallery/editor-rtl.css 749 B 0 B
build/block-library/blocks/gallery/editor.css 750 B 0 B
build/block-library/blocks/gallery/style-rtl.css 1.17 kB 0 B
build/block-library/blocks/gallery/style.css 1.17 kB 0 B
build/block-library/blocks/group/editor-rtl.css 433 B 0 B
build/block-library/blocks/group/editor.css 432 B 0 B
build/block-library/blocks/group/style-rtl.css 190 B 0 B
build/block-library/blocks/group/style.css 190 B 0 B
build/block-library/blocks/heading/editor-rtl.css 248 B 0 B
build/block-library/blocks/heading/editor.css 248 B 0 B
build/block-library/blocks/heading/style-rtl.css 212 B 0 B
build/block-library/blocks/heading/style.css 212 B 0 B
build/block-library/blocks/html/editor-rtl.css 384 B 0 B
build/block-library/blocks/html/editor.css 385 B 0 B
build/block-library/blocks/image/editor-rtl.css 801 B 0 B
build/block-library/blocks/image/editor.css 800 B 0 B
build/block-library/blocks/image/style-rtl.css 569 B 0 B
build/block-library/blocks/image/style.css 570 B 0 B
build/block-library/blocks/latest-comments/editor-rtl.css 277 B 0 B
build/block-library/blocks/latest-comments/editor.css 275 B 0 B
build/block-library/blocks/latest-comments/style-rtl.css 382 B 0 B
build/block-library/blocks/latest-comments/style.css 382 B 0 B
build/block-library/blocks/latest-posts/editor-rtl.css 254 B 0 B
build/block-library/blocks/latest-posts/editor.css 254 B 0 B
build/block-library/blocks/latest-posts/style-rtl.css 634 B 0 B
build/block-library/blocks/latest-posts/style.css 634 B 0 B
build/block-library/blocks/list/editor-rtl.css 203 B 0 B
build/block-library/blocks/list/editor.css 203 B 0 B
build/block-library/blocks/list/style-rtl.css 201 B 0 B
build/block-library/blocks/list/style.css 201 B 0 B
build/block-library/blocks/media-text/editor-rtl.css 311 B 0 B
build/block-library/blocks/media-text/editor.css 311 B 0 B
build/block-library/blocks/media-text/style-rtl.css 642 B 0 B
build/block-library/blocks/media-text/style.css 640 B 0 B
build/block-library/blocks/more/editor-rtl.css 545 B 0 B
build/block-library/blocks/more/editor.css 545 B 0 B
build/block-library/blocks/navigation-link/editor-rtl.css 503 B 0 B
build/block-library/blocks/navigation-link/editor.css 504 B 0 B
build/block-library/blocks/navigation-link/style-rtl.css 805 B 0 B
build/block-library/blocks/navigation-link/style.css 803 B 0 B
build/block-library/blocks/navigation/editor-rtl.css 1.38 kB 0 B
build/block-library/blocks/navigation/editor.css 1.38 kB 0 B
build/block-library/blocks/navigation/style-rtl.css 274 B 0 B
build/block-library/blocks/navigation/style.css 274 B 0 B
build/block-library/blocks/nextpage/editor-rtl.css 507 B 0 B
build/block-library/blocks/nextpage/editor.css 507 B 0 B
build/block-library/blocks/paragraph/editor-rtl.css 236 B 0 B
build/block-library/blocks/paragraph/editor.css 236 B 0 B
build/block-library/blocks/paragraph/style-rtl.css 351 B 0 B
build/block-library/blocks/paragraph/style.css 352 B 0 B
build/block-library/blocks/post-author/editor-rtl.css 329 B 0 B
build/block-library/blocks/post-author/editor.css 329 B 0 B
build/block-library/blocks/post-author/style-rtl.css 303 B 0 B
build/block-library/blocks/post-author/style.css 303 B 0 B
build/block-library/blocks/post-comments-form/style-rtl.css 358 B 0 B
build/block-library/blocks/post-comments-form/style.css 358 B 0 B
build/block-library/blocks/post-content/editor-rtl.css 262 B 0 B
build/block-library/blocks/post-content/editor.css 262 B 0 B
build/block-library/blocks/post-excerpt/editor-rtl.css 206 B 0 B
build/block-library/blocks/post-excerpt/editor.css 206 B 0 B
build/block-library/blocks/post-featured-image/editor-rtl.css 453 B 0 B
build/block-library/blocks/post-featured-image/editor.css 453 B 0 B
build/block-library/blocks/post-featured-image/style-rtl.css 223 B 0 B
build/block-library/blocks/post-featured-image/style.css 223 B 0 B
build/block-library/blocks/preformatted/style-rtl.css 193 B 0 B
build/block-library/blocks/preformatted/style.css 193 B 0 B
build/block-library/blocks/pullquote/editor-rtl.css 304 B 0 B
build/block-library/blocks/pullquote/editor.css 304 B 0 B
build/block-library/blocks/pullquote/style-rtl.css 428 B 0 B
build/block-library/blocks/pullquote/style.css 428 B 0 B
build/block-library/blocks/query-loop/editor-rtl.css 217 B 0 B
build/block-library/blocks/query-loop/editor.css 216 B 0 B
build/block-library/blocks/query-loop/style-rtl.css 427 B 0 B
build/block-library/blocks/query-loop/style.css 429 B 0 B
build/block-library/blocks/query/editor-rtl.css 279 B 0 B
build/block-library/blocks/query/editor.css 279 B 0 B
build/block-library/blocks/quote/editor-rtl.css 195 B 0 B
build/block-library/blocks/quote/editor.css 195 B 0 B
build/block-library/blocks/quote/style-rtl.css 284 B 0 B
build/block-library/blocks/quote/style.css 285 B 0 B
build/block-library/blocks/rss/editor-rtl.css 307 B 0 B
build/block-library/blocks/rss/editor.css 309 B 0 B
build/block-library/blocks/rss/style-rtl.css 394 B 0 B
build/block-library/blocks/rss/style.css 393 B 0 B
build/block-library/blocks/search/editor-rtl.css 285 B 0 B
build/block-library/blocks/search/editor.css 285 B 0 B
build/block-library/blocks/search/style-rtl.css 454 B 0 B
build/block-library/blocks/search/style.css 456 B 0 B
build/block-library/blocks/separator/editor-rtl.css 229 B 0 B
build/block-library/blocks/separator/editor.css 229 B 0 B
build/block-library/blocks/separator/style-rtl.css 352 B 0 B
build/block-library/blocks/separator/style.css 352 B 0 B
build/block-library/blocks/shortcode/editor-rtl.css 603 B 0 B
build/block-library/blocks/shortcode/editor.css 603 B 0 B
build/block-library/blocks/site-logo/editor-rtl.css 321 B 0 B
build/block-library/blocks/site-logo/editor.css 321 B 0 B
build/block-library/blocks/site-logo/style-rtl.css 238 B 0 B
build/block-library/blocks/site-logo/style.css 238 B 0 B
build/block-library/blocks/social-link/editor-rtl.css 283 B 0 B
build/block-library/blocks/social-link/editor.css 283 B 0 B
build/block-library/blocks/social-links/editor-rtl.css 811 B 0 B
build/block-library/blocks/social-links/editor.css 810 B 0 B
build/block-library/blocks/social-links/style-rtl.css 1.44 kB 0 B
build/block-library/blocks/social-links/style.css 1.44 kB 0 B
build/block-library/blocks/spacer/editor-rtl.css 391 B 0 B
build/block-library/blocks/spacer/editor.css 391 B 0 B
build/block-library/blocks/spacer/style-rtl.css 184 B 0 B
build/block-library/blocks/spacer/style.css 184 B 0 B
build/block-library/blocks/subhead/editor-rtl.css 223 B 0 B
build/block-library/blocks/subhead/editor.css 223 B 0 B
build/block-library/blocks/subhead/style-rtl.css 210 B 0 B
build/block-library/blocks/subhead/style.css 210 B 0 B
build/block-library/blocks/table/editor-rtl.css 593 B 0 B
build/block-library/blocks/table/editor.css 593 B 0 B
build/block-library/blocks/table/style-rtl.css 501 B 0 B
build/block-library/blocks/table/style.css 501 B 0 B
build/block-library/blocks/tag-cloud/editor-rtl.css 237 B 0 B
build/block-library/blocks/tag-cloud/editor.css 235 B 0 B
build/block-library/blocks/tag-cloud/style-rtl.css 221 B 0 B
build/block-library/blocks/tag-cloud/style.css 221 B 0 B
build/block-library/blocks/template-part/editor-rtl.css 714 B 0 B
build/block-library/blocks/template-part/editor.css 714 B 0 B
build/block-library/blocks/text-columns/editor-rtl.css 220 B 0 B
build/block-library/blocks/text-columns/editor.css 220 B 0 B
build/block-library/blocks/text-columns/style-rtl.css 283 B 0 B
build/block-library/blocks/text-columns/style.css 283 B 0 B
build/block-library/blocks/verse/editor-rtl.css 194 B 0 B
build/block-library/blocks/verse/editor.css 194 B 0 B
build/block-library/blocks/verse/style-rtl.css 215 B 0 B
build/block-library/blocks/verse/style.css 215 B 0 B
build/block-library/blocks/video/editor-rtl.css 617 B 0 B
build/block-library/blocks/video/editor.css 617 B 0 B
build/block-library/blocks/video/style-rtl.css 303 B 0 B
build/block-library/blocks/video/style.css 304 B 0 B
build/block-library/common-rtl.css 1.01 kB 0 B
build/block-library/common.css 1.01 kB 0 B
build/block-library/theme-rtl.css 860 B 0 B
build/block-library/theme.css 860 B 0 B
build/block-serialization-default-parser/index.js 1.88 kB 0 B
build/block-serialization-spec-parser/index.js 3.06 kB 0 B
build/blocks/index.js 48.1 kB 0 B
build/components/index.js 173 kB 0 B
build/components/style-rtl.css 15.5 kB 0 B
build/components/style.css 15.5 kB 0 B
build/compose/index.js 11.2 kB 0 B
build/data-controls/index.js 829 B 0 B
build/data/index.js 8.97 kB 0 B
build/date/index.js 31.8 kB 0 B
build/deprecated/index.js 768 B 0 B
build/dom-ready/index.js 571 B 0 B
build/dom/index.js 4.95 kB 0 B
build/edit-navigation/index.js 11.1 kB 0 B
build/edit-navigation/style-rtl.css 938 B 0 B
build/edit-navigation/style.css 944 B 0 B
build/edit-post/index.js 306 kB 0 B
build/edit-widgets/index.js 23.6 kB 0 B
build/editor/editor-styles-rtl.css 476 B 0 B
build/editor/editor-styles.css 478 B 0 B
build/editor/index.js 42.3 kB 0 B
build/editor/style-rtl.css 3.89 kB 0 B
build/editor/style.css 3.89 kB 0 B
build/element/index.js 4.62 kB 0 B
build/escape-html/index.js 735 B 0 B
build/format-library/index.js 6.75 kB 0 B
build/format-library/style-rtl.css 620 B 0 B
build/format-library/style.css 621 B 0 B
build/hooks/index.js 2.27 kB 0 B
build/html-entities/index.js 623 B 0 B
build/is-shallow-equal/index.js 698 B 0 B
build/keyboard-shortcuts/index.js 2.54 kB 0 B
build/keycodes/index.js 1.94 kB 0 B
build/list-reusable-blocks/index.js 3.15 kB 0 B
build/list-reusable-blocks/style-rtl.css 629 B 0 B
build/list-reusable-blocks/style.css 628 B 0 B
build/media-utils/index.js 5.31 kB 0 B
build/notices/index.js 1.86 kB 0 B
build/nux/index.js 3.42 kB 0 B
build/nux/style-rtl.css 731 B 0 B
build/nux/style.css 727 B 0 B
build/plugins/index.js 2.54 kB 0 B
build/primitives/index.js 1.43 kB 0 B
build/priority-queue/index.js 789 B 0 B
build/redux-routine/index.js 2.84 kB 0 B
build/reusable-blocks/index.js 2.91 kB 0 B
build/server-side-render/index.js 2.77 kB 0 B
build/shortcode/index.js 1.7 kB 0 B
build/token-list/index.js 1.27 kB 0 B
build/url/index.js 3.01 kB 0 B
build/viewport/index.js 1.85 kB 0 B
build/warning/index.js 1.14 kB 0 B
build/wordcount/index.js 1.22 kB 0 B

compressed-size-action

@kevin940726 kevin940726 force-pushed the add/widgets-screen-e2e branch from da4ad6a to 5b3d1c8 Compare January 13, 2021 06:42
Copy link
Member

@noisysocks noisysocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! I think we should look at snapshotting the actual widget entities (getWidgets()) instead of the block representation (getWidgetAreas()). Other than that, this looks good as something that we can merge and iterate on in follow-up PRs.

packages/e2e-tests/specs/experiments/widgets.test.js Outdated Show resolved Hide resolved
} );

beforeAll( async () => {
// TODO: Ideally we can bundle our test theme directly in the repo.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's some prior art here with the tt1-blocks theme which is used to test FSE.

"themes": [ "WordPress/theme-experiments/tt1-blocks" ],

We can look into it later, though. It's more important that we get some tests for widgets in right now 🙂

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm aware of that, but figured we should do it in a following PR 👍 .

packages/e2e-tests/specs/experiments/widgets.test.js Outdated Show resolved Hide resolved
await activateTheme( 'twentytwentyone' );
} );

async function getParagraphBlockInGlobalInserter() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Why are some of these helper functions at the top here and some at the bottom of the file? 😀

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a pattern more aligned to my mental model 😆 . Functions here are regular utility functions, but functions at the bottom should eventually be moved to the e2e-test-utils package once they're stable.

packages/e2e-tests/specs/experiments/widgets.test.js Outdated Show resolved Hide resolved
packages/e2e-tests/specs/experiments/widgets.test.js Outdated Show resolved Hide resolved
@kevin940726 kevin940726 marked this pull request as ready for review January 14, 2021 04:39
Copy link
Member

@noisysocks noisysocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a great start! Nice work.

Comment on lines +69 to +72
// TODO: Probably need a more accessible way to select this, maybe a test ID or data attribute.
const insertionPointIndicator = await page.$(
'.block-editor-block-list__insertion-point-indicator'
);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #28185.

Comment on lines +89 to +92
// FIXME: The insertion point indicator is not showing when the widget area has no blocks.
// await expectInsertionPointIndicatorToBeBelowLastBlock(
// firstWidgetArea
// );
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #28123.

Also, unless there are multiple widgets in a sidebar area, I'm unable to get the in-between inserter to appear.

@kevin940726 kevin940726 merged commit 90920ec into master Jan 14, 2021
@kevin940726 kevin940726 deleted the add/widgets-screen-e2e branch January 14, 2021 09:22
@github-actions github-actions bot added this to the Gutenberg 9.9 milestone Jan 14, 2021

beforeAll( async () => {
// TODO: Ideally we can bundle our test theme directly in the repo.
await activateTheme( 'twentytwenty' );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why specifically TwentyTwenty?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just happens to have 2 widget areas so that we can test DnD and some other stuffs more easily 😅 .

@youknowriad
Copy link
Contributor

Thanks for the test

Should we move it to specs/experimental though? That separation is used to know which tests are stable (can run on WP core without the plugin) or not.

@kevin940726 kevin940726 mentioned this pull request Jan 18, 2021
6 tasks
@kevin940726
Copy link
Member Author

@youknowriad Ahh got it, I didn't know what it means and where to put this, and seems like most of us don't know 😆 : #28160 (comment).

@jasmussen jasmussen mentioned this pull request Oct 13, 2021
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Feature] Widgets Screen The block-based screen that replaced widgets.php. [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Enhancement A suggestion for improvement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Widgets editor: Add more E2E tests
4 participants