Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E Utils: Use frameLocator for retrieving editor canvas #54911

Merged
merged 33 commits into from
Oct 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
687d6ea
Try using frameLocator for retrieving canvas
WunderBart Sep 28, 2023
a755917
Replace canvas.waitForFunction with expec.poll
WunderBart Sep 28, 2023
0260da0
Update e2e utils to use frameLocator for editor canvas
WunderBart Sep 28, 2023
e41131e
Migrate tests to frameLocator API
WunderBart Sep 28, 2023
405b447
Fix image spec
WunderBart Sep 29, 2023
5a9bcfa
Fix navigation-colors spec
WunderBart Sep 29, 2023
51a3a75
Fix paragraph spec
WunderBart Sep 29, 2023
d7d05d1
Fix draggable-blocks spec
WunderBart Sep 29, 2023
683f209
Fix inserting blocks spec
WunderBart Oct 2, 2023
53fcf97
Fix wp editor meta box spec
WunderBart Oct 2, 2023
5cdaf1d
Don't wait for editor when creating post
WunderBart Oct 2, 2023
17ff61f
Create switchToLegacyCanvas util
WunderBart Oct 2, 2023
730e9d8
Fix a11y spec
WunderBart Oct 2, 2023
08c9b6f
Use switchToLegacyCanvas in inserting blocks spec
WunderBart Oct 2, 2023
6dcda8a
Fix copy cut paste spec
WunderBart Oct 2, 2023
9378166
Fix keep styles on block transforms spec
WunderBart Oct 2, 2023
5a8f3ef
Fix list view spec
WunderBart Oct 2, 2023
1262f39
Fix multi block selection spec
WunderBart Oct 2, 2023
9ca19cd
Fix template part spec
WunderBart Oct 2, 2023
c5756bb
Fix customizing widgets spec
WunderBart Oct 2, 2023
17fc072
Merge remote-tracking branch 'origin' into refactor/e2e-use-frame-loc…
WunderBart Oct 3, 2023
eacc4fd
Fix footnotes spec
WunderBart Oct 3, 2023
5bf6ffd
Use switchToLegacyCanvas in classic spec
WunderBart Oct 3, 2023
7691627
Remove some leftovers
WunderBart Oct 3, 2023
7851a01
Revert to legacy canvas to pass CI
WunderBart Oct 3, 2023
0c9651a
Fix writing flow spec
WunderBart Oct 3, 2023
efd7918
Ensure editor utils self-validate
WunderBart Oct 4, 2023
72e4fb5
Update nonce spec
WunderBart Oct 4, 2023
b0c82ea
Fix a11y region navigation spec
WunderBart Oct 4, 2023
b84db19
Merge remote-tracking branch 'origin' into refactor/e2e-use-frame-loc…
WunderBart Oct 4, 2023
6efac5c
Fix global styles sidebar spec
WunderBart Oct 4, 2023
bab65f8
Address flaky splitting merging specs
WunderBart Oct 5, 2023
fd96c75
Don't fall back to page if canvas isn't iframed
WunderBart Oct 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 6 additions & 14 deletions packages/e2e-test-utils-playwright/src/admin/create-new-post.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,26 +30,18 @@ export async function createNewPost( {

await this.visitAdminPage( 'post-new.php', query );

// Wait for both iframed and non-iframed canvas and resolve once the
// currently available one is ready. To make this work, we need an inner
// legacy canvas selector that is unavailable directly when the canvas is
// iframed.
await Promise.any( [
this.page.locator( '.wp-block-post-content' ).waitFor(),
this.page
.frameLocator( '[name=editor-canvas]' )
.locator( 'body > *' )
.first()
.waitFor(),
] );
Comment on lines -33 to -44
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep this. The missing lazy evaluation from editor.canvas wasn't the only reason for this safeguard.

Why?

  • Expectation that editor canvas is loaded when performing an action after the createNewPost call. Inherited from Puppeteer utils. It could count as a breaking change.
  • Some of the test patterns don't use locators for creating blocks. See examples below.

Examples

// Insert a block using `editor` utils. Was flaky before https://github.com/WordPress/gutenberg/pull/51824.
await admin.createNewPost();
await editor.insertBlock( { name: 'core/heading' } );

// Add blocks using writing flow "shortcut":
await admin.createNewPost();
await page.keyboard.press( 'Enter' );
await page.keyboard.type( 'Content' );

The last pattern was very common in Puppeteer but became flaky after #48286 (before #51824), so we had to switch to editor.canvas.click( 'role=button[name="Add default block"i]' ).

Copy link
Member Author

@WunderBart WunderBart Oct 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Believing that the entire editor is immediately ready upon invoking callingNewPost(), especially considering editor's dynamic nature, can be a risky assumption. Instead of making such assumptions, we should directly ensure that the resources we intend to use are available at the time of reference in the context where the resource is accessed.

From the sample you shared, it seems more logical to adjust the insertBlock() function.
Rather than having:

async function insertBlock(
	this: Editor,
	blockRepresentation: BlockRepresentation
) {
	await this.page.evaluate( ( _blockRepresentation ) => {

		//...

		window.wp.data.dispatch( 'core/block-editor' ).insertBlock( block );
	}, blockRepresentation );

I propose this version:

async function insertBlock(
	this: Editor,
	blockRepresentation: BlockRepresentation
) {
	await this.page.waitForFunction( ( _blockRepresentation ) => {
		if ( ! window?.wp?.data ) {
			return false;
		}

		//...

		window.wp.data.dispatch( 'core/block-editor' ).insertBlock( block );
	}, blockRepresentation );

In this alternative, the function waits until the window.wp.data resource is ready before proceeding. The immediate use of evaluate() presupposes the availability of window.wp.data, effectively moving the responsibility of ensuring its presence outside this function. To illustrate:

await admin.createNewPost();
await editor.insertBlock( { name: 'core/heading' } );

await this.page.reload();
await editor.insertBlock( { name: 'core/heading' } );

If the responsibility to check resource availability lies within createNewPost(), the second insertBlock() call could be unstable. However, if insertBlock() self-validates that its required resources are ready, we ensure a more robust outcome.

Does that perspective make sense?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that perspective make sense?

For me, it does. I don't feel a strong attachment to previous behavior 😄

As I mentioned, this changes the "expected" behavior of createNewPost(), and not everyone actively follows changes to the e2e test environment.

I like your suggestion for improving the insertBlock utility 🚀

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for the second example you provided, it involves some ambiguity because we assume the cursor is in the correct spot, but we don't really know what that spot is. Instead of shifting the burden of ensuring the cursor is where it should be to createNewPost() (which is not even the case, because we'd only wait for the editor body to be visible, which doesn't ensure cursor position!), I propose adding a step that explicitly waits for the cursor to be in the right spot:

await admin.createNewPost();
await expect(
  editor.canvas.getByRole( 'textbox', { name: 'Add title' } )
).toBeFocused();

await page.keyboard.press( 'Enter' );
await page.keyboard.type( 'Content' );

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One DX benefit of removing this is it cleans up a little noise in the trace viewer that is caused by one of these locators timing out.


await this.page.evaluate( ( welcomeGuide ) => {
await this.page.waitForFunction( ( welcomeGuide ) => {
if ( ! window?.wp?.data?.dispatch ) {
return false;
}
window.wp.data
.dispatch( 'core/preferences' )
.set( 'core/edit-post', 'welcomeGuide', welcomeGuide );

window.wp.data
.dispatch( 'core/preferences' )
.set( 'core/edit-post', 'fullscreenMode', false );

return true;
}, showWelcomeGuide );
}
4 changes: 4 additions & 0 deletions packages/e2e-test-utils-playwright/src/editor/get-blocks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ type Block = {
* @return The blocks.
*/
export async function getBlocks( this: Editor, { full = false } = {} ) {
await this.page.waitForFunction(
() => window?.wp?.blocks && window?.wp?.data
);

return await this.page.evaluate(
( [ _full ] ) => {
// Remove other unpredictable properties like clientId from blocks for testing purposes.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ import type { Editor } from './index';
* @return Promise resolving with post content markup.
*/
export async function getEditedPostContent( this: Editor ) {
await this.page.waitForFunction( () => window?.wp?.data );

return await this.page.evaluate( () =>
window.wp.data.select( 'core/editor' ).getEditedPostContent()
);
Expand Down
15 changes: 12 additions & 3 deletions packages/e2e-test-utils-playwright/src/editor/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
/**
* External dependencies
*/
import type { Browser, Page, BrowserContext, Frame } from '@playwright/test';
import type {
Browser,
Page,
BrowserContext,
FrameLocator,
} from '@playwright/test';

/**
* Internal dependencies
Expand All @@ -19,6 +24,7 @@ import { setContent } from './set-content';
import { showBlockToolbar } from './show-block-toolbar';
import { saveSiteEditorEntities } from './site-editor';
import { setIsFixedToolbar } from './set-is-fixed-toolbar';
import { switchToLegacyCanvas } from './switch-to-legacy-canvas';
import { transformBlockTo } from './transform-block-to';

type EditorConstructorProps = {
Expand All @@ -36,8 +42,8 @@ export class Editor {
this.browser = this.context.browser()!;
}

get canvas(): Frame | Page {
return this.page.frame( 'editor-canvas' ) || this.page;
get canvas(): FrameLocator {
return this.page.frameLocator( '[name="editor-canvas"]' );
}

/** @borrows clickBlockOptionsMenuItem as this.clickBlockOptionsMenuItem */
Expand Down Expand Up @@ -72,6 +78,9 @@ export class Editor {
/** @borrows setIsFixedToolbar as this.setIsFixedToolbar */
setIsFixedToolbar: typeof setIsFixedToolbar =
setIsFixedToolbar.bind( this );
/** @borrows switchToLegacyCanvas as this.switchToLegacyCanvas */
switchToLegacyCanvas: typeof switchToLegacyCanvas =
switchToLegacyCanvas.bind( this );
/** @borrows transformBlockTo as this.transformBlockTo */
transformBlockTo: typeof transformBlockTo = transformBlockTo.bind( this );
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ async function insertBlock(
this: Editor,
blockRepresentation: BlockRepresentation
) {
await this.page.waitForFunction(
() => window?.wp?.blocks && window?.wp?.data
);

await this.page.evaluate( ( _blockRepresentation ) => {
function recursiveCreateBlock( {
name,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ import type { Editor } from './index';
* @param html Serialized block HTML.
*/
async function setContent( this: Editor, html: string ) {
await this.page.waitForFunction(
() => window?.wp?.blocks && window?.wp?.data
);

await this.page.evaluate( ( _html ) => {
const blocks = window.wp.blocks.parse( _html );

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import type { Editor } from './index';
* @param isFixed Boolean value true/false for on/off.
*/
export async function setIsFixedToolbar( this: Editor, isFixed: boolean ) {
await this.page.waitForFunction( () => window?.wp?.data );

await this.page.evaluate( ( _isFixed ) => {
window.wp.data
.dispatch( 'core/preferences' )
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/**
* Internal dependencies
*/
import type { Editor } from './index';

/**
* Switches to legacy (non-iframed) canvas.
*
* @param this
*/
export async function switchToLegacyCanvas( this: Editor ) {
await this.page.waitForFunction( () => window?.wp?.blocks );

await this.page.evaluate( () => {
window.wp.blocks.registerBlockType( 'test/v2', {
apiVersion: '2',
title: 'test',
} );
} );
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ import type { Editor } from './index';
* @param name Block name.
*/
export async function transformBlockTo( this: Editor, name: string ) {
await this.page.waitForFunction(
() => window?.wp?.blocks && window?.wp?.data
);

await this.page.evaluate(
( [ blockName ] ) => {
const clientIds = window.wp.data
Expand Down
46 changes: 23 additions & 23 deletions packages/e2e-test-utils-playwright/src/page-utils/drag-files.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,29 +54,6 @@ async function dragFiles(
} )
);

const dataTransfer = await this.page.evaluateHandle(
async ( _fileObjects ) => {
const dt = new DataTransfer();
const fileInstances = await Promise.all(
_fileObjects.map( async ( fileObject ) => {
const blob = await fetch(
`data:${ fileObject.mimeType };base64,${ fileObject.base64 }`
).then( ( res ) => res.blob() );
return new File( [ blob ], fileObject.name, {
type: fileObject.mimeType ?? undefined,
} );
} )
);

fileInstances.forEach( ( file ) => {
dt.items.add( file );
} );

return dt;
},
fileObjects
);

// CDP doesn't actually support dragging files, this is only a _good enough_
// dummy data so that it will correctly send the relevant events.
const dragData = {
Expand Down Expand Up @@ -159,6 +136,29 @@ async function dragFiles(
throw new Error( 'Element not found.' );
}

const dataTransfer = await locator.evaluateHandle(
async ( _node, _fileObjects ) => {
const dt = new DataTransfer();
const fileInstances = await Promise.all(
_fileObjects.map( async ( fileObject: any ) => {
const blob = await fetch(
`data:${ fileObject.mimeType };base64,${ fileObject.base64 }`
).then( ( res ) => res.blob() );
return new File( [ blob ], fileObject.name, {
type: fileObject.mimeType ?? undefined,
} );
} )
);

fileInstances.forEach( ( file ) => {
dt.items.add( file );
} );

return dt;
},
fileObjects
);

await locator.dispatchEvent( 'drop', { dataTransfer } );

await cdpSession.detach();
Expand Down
4 changes: 3 additions & 1 deletion test/e2e/specs/editor/blocks/buttons.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ test.describe( 'Buttons', () => {
editor,
page,
} ) => {
await editor.canvas.click( 'role=button[name="Add default block"i]' );
await editor.canvas
.locator( 'role=button[name="Add default block"i]' )
.click();
Comment on lines +33 to +35
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sidenote: This is a really common pattern; I wonder if we should extract it into a utility method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
await editor.canvas
.locator( 'role=button[name="Add default block"i]' )
.click();
await editor.canvas
.getByRole( 'button', {
name: "Add default block"
})
.click();

Nit: I have previously been advised to prefer getByRole wherever possible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably not worth refactoring old code for this standard, especially if we plan to extract this in helper.

Both ways are fine, but using RTL-style locators is preferred.

await page.keyboard.type( '/buttons' );
await page.keyboard.press( 'Enter' );
await page.keyboard.type( 'Content' );
Expand Down
16 changes: 3 additions & 13 deletions test/e2e/specs/editor/blocks/classic.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,10 @@ test.use( {
} );

test.describe( 'Classic', () => {
test.beforeEach( async ( { admin, page } ) => {
test.beforeEach( async ( { admin, editor } ) => {
await admin.createNewPost();
// To do: run with iframe.
await page.evaluate( () => {
window.wp.blocks.registerBlockType( 'test/v2', {
apiVersion: '2',
title: 'test',
} );
} );
await editor.switchToLegacyCanvas();
} );

test.afterAll( async ( { requestUtils } ) => {
Expand Down Expand Up @@ -134,12 +129,7 @@ test.describe( 'Classic', () => {
await page.unroute( '**' );

// To do: run with iframe.
await page.evaluate( () => {
window.wp.blocks.registerBlockType( 'test/v2', {
apiVersion: '2',
title: 'test',
} );
} );
await editor.switchToLegacyCanvas();

const errors = [];
page.on( 'pageerror', ( exception ) => {
Expand Down
4 changes: 3 additions & 1 deletion test/e2e/specs/editor/blocks/code.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ test.describe( 'Code', () => {
editor,
page,
} ) => {
await editor.canvas.click( 'role=button[name="Add default block"i]' );
await editor.canvas
.locator( 'role=button[name="Add default block"i]' )
.click();
await page.keyboard.type( '```' );
await page.keyboard.press( 'Enter' );
await page.keyboard.type( '<?php' );
Expand Down
6 changes: 3 additions & 3 deletions test/e2e/specs/editor/blocks/comments.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -165,9 +165,9 @@ test.describe( 'Comments', () => {
await expect( warning ).toBeVisible();
await expect( placeholder ).toBeVisible();

await editor.canvas.click(
'role=button[name="Switch to editable mode"i]'
);
await editor.canvas
.locator( 'role=button[name="Switch to editable mode"i]' )
.click();

const commentTemplate = editor.canvas.locator(
'role=document[name="Block: Comment Template"i]'
Expand Down
8 changes: 6 additions & 2 deletions test/e2e/specs/editor/blocks/gallery.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,9 @@ test.describe( 'Gallery', () => {
plainText: `[gallery ids="${ uploadedMedia.id }"]`,
} );

await editor.canvas.click( 'role=button[name="Add default block"i]' );
await editor.canvas
.locator( 'role=button[name="Add default block"i]' )
.click();
await pageUtils.pressKeys( 'primary+v' );

const img = editor.canvas.locator(
Expand Down Expand Up @@ -204,7 +206,9 @@ test.describe( 'Gallery', () => {
} ) => {
await admin.createNewPost();
await editor.insertBlock( { name: 'core/gallery' } );
await editor.canvas.click( 'role=button[name="Media Library"i]' );
await editor.canvas
.locator( 'role=button[name="Media Library"i]' )
.click();

const mediaLibrary = page.locator(
'role=dialog[name="Create gallery"i]'
Expand Down
30 changes: 19 additions & 11 deletions test/e2e/specs/editor/blocks/group.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,11 @@ test.describe( 'Group', () => {
);

// Select the default, selected Group layout from the variation picker.
await editor.canvas.click(
'role=button[name="Group: Gather blocks in a container."i]'
);
await editor.canvas
.locator(
'role=button[name="Group: Gather blocks in a container."i]'
)
.click();

expect( await editor.getEditedPostContent() ).toMatchSnapshot();
} );
Expand All @@ -40,17 +42,21 @@ test.describe( 'Group', () => {
editor,
page,
} ) => {
await editor.canvas.click( 'role=button[name="Add default block"i]' );
await editor.canvas
.locator( 'role=button[name="Add default block"i]' )
.click();
await page.keyboard.type( '/group' );
await expect(
page.locator( 'role=option[name="Group"i][selected]' )
).toBeVisible();
await page.keyboard.press( 'Enter' );

// Select the default, selected Group layout from the variation picker.
await editor.canvas.click(
'role=button[name="Group: Gather blocks in a container."i]'
);
await editor.canvas
.locator(
'role=button[name="Group: Gather blocks in a container."i]'
)
.click();

expect( await editor.getEditedPostContent() ).toMatchSnapshot();
} );
Expand All @@ -60,10 +66,12 @@ test.describe( 'Group', () => {
page,
} ) => {
await editor.insertBlock( { name: 'core/group' } );
await editor.canvas.click(
'button[aria-label="Group: Gather blocks in a container."]'
);
await editor.canvas.click( 'role=button[name="Add block"i]' );
await editor.canvas
.locator(
'button[aria-label="Group: Gather blocks in a container."]'
)
.click();
await editor.canvas.locator( 'role=button[name="Add block"i]' ).click();
await page.click(
'role=listbox[name="Blocks"i] >> role=option[name="Paragraph"i]'
);
Expand Down
Loading
Loading