E2e test fixtures #16061

PeterYinusa · 2022-10-03T19:05:54Z

Explanation

Previously multiple e2e tests were coupled to a single state.json file.

This PR decouples the state files, and provides an easy to use api to create fixture state, enabling more granular control of each individual tests state.

There are two initial state/fixture objects, default fixture and onboarding fixture.
The onboarding fixture provides the state of a new user pre onboarding
The default fixture provides the state of an existing user post onboarding

More Information

Screenshots/Screencaps

Before

After

Manual Testing Steps

Pre-Merge Checklist

PR template is filled out
IF this PR fixes a bug, a test that would have caught the bug has been added
PR is linked to the appropriate GitHub issue
PR has been added to the appropriate release Milestone

+ If there are functional changes:

Manual testing complete & passed
"Extension QA Board" label has been applied

github-actions · 2022-10-03T19:06:06Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

PeterYinusa · 2022-10-03T19:08:32Z

test/e2e/benchmark.js

-    metrics = await driver.collectMetrics();
-  });
+  await withFixtures(
+    { fixtures: new FixtureBuilder().build() },


Only the fixture has changed, the rest is just lint formatting

PeterYinusa · 2022-10-03T19:11:38Z

test/e2e/helpers.js

@@ -198,26 +198,6 @@ const getWindowHandles = async (driver, handlesCount) => {
  return { extension, dapp, popup };
 };

-const connectDappWithExtensionPopup = async (driver) => {


We use the FixtureBuilder to add the required state for dapp connections

PeterYinusa · 2022-10-03T19:12:11Z

test/e2e/lavamoat-stats.js

-      metrics = parsedLogs.map((pl) => JSON.parse(pl));
-    });
+    await withFixtures(
+      { fixtures: new FixtureBuilder().build() },


Only the fixture has changed, the rest is just lint formatting

PeterYinusa · 2022-10-03T19:12:30Z

test/e2e/mv3-perf-stats/init-load-stats.js

-      await driver.delay(1000);
-      const logs = await driver.checkBrowserForLavamoatLogs();
+    await withFixtures(
+      { fixtures: new FixtureBuilder().build() },


Only the fixture has changed, the rest is just lint formatting

PeterYinusa · 2022-10-03T19:12:40Z

test/e2e/mv3-stats.js

-      await driver.delay(1000);
-      const logs = await driver.checkBrowserForLavamoatLogs();
+    await withFixtures(
+      { fixtures: new FixtureBuilder().build() },


Only the fixture has changed, the rest is just lint formatting

PeterYinusa · 2022-10-03T19:19:50Z

test/e2e/tests/dapp-interactions.spec.js

@@ -34,8 +31,7 @@ describe('Dapp interactions', function () {
        await driver.fill('#password', 'correct horse battery staple');
        await driver.press('#password', driver.Key.ENTER);

-        // Connect to Dapp0
-        await connectDappWithExtensionPopup(driver, 0);


We do not actually need to connect to the dapp to click the add ethereum chain button

PeterYinusa · 2022-10-03T19:20:10Z

test/e2e/tests/dapp-interactions.spec.js

@@ -78,8 +76,7 @@ describe('Dapp interactions', function () {
        await driver.fill('#password', 'correct horse battery staple');
        await driver.press('#password', driver.Key.ENTER);

-        // Connect to Dapp0
-        await connectDappWithExtensionPopup(driver, 0);


We can use the fixture to connect to the dapp

PeterYinusa · 2022-10-03T19:21:21Z

test/e2e/tests/edit-gas-fee.spec.js

@@ -34,7 +39,7 @@ describe('Editing Confirm Transaction', function () {
          '.currency-display-component__text',
        );
        const transactionAmount = transactionAmounts[0];
-        assert.equal(await transactionAmount.getText(), '2.2');
+        assert.equal(await transactionAmount.getText(), '1');


The value has changed as we are now reusing an existing transaction

PeterYinusa · 2022-10-03T19:21:50Z

test/e2e/tests/edit-gas-fee.spec.js

@@ -112,7 +122,7 @@ describe('Editing Confirm Transaction', function () {
          '.currency-display-component__text',
        );
        const transactionAmount = transactionAmounts[0];
-        assert.equal(await transactionAmount.getText(), '2.2');
+        assert.equal(await transactionAmount.getText(), '1');


The value has changed as we are now reusing an existing transaction

PeterYinusa · 2022-10-03T19:22:29Z

test/e2e/tests/edit-gas-fee.spec.js

@@ -194,7 +209,7 @@ describe('Editing Confirm Transaction', function () {
        await driver.press('#password', driver.Key.ENTER);

        // open dapp and connect
-        await connectDappWithExtensionPopup(driver);


We can use the fixture to connect to the dapp

PeterYinusa · 2022-10-03T19:23:03Z

test/e2e/tests/from-import-ui.spec.js

@@ -315,7 +344,6 @@ describe('MetaMask Import UI', function () {
        const importJsonFile = path.join(
          __dirname,
          '..',
-          'fixtures',


moved this file out of the fixture folder as its not a fixture

PeterYinusa · 2022-10-04T10:40:48Z

test/e2e/snaps/test-snap-bip-44.spec.js

@@ -39,26 +42,9 @@ describe('Test Snap bip-44', function () {
        // connect the snap
        await driver.clickElement('#connectBip44');

-        // switch to metamask extension and click connect


Connect to the snap test dapp in the fixture

PeterYinusa · 2022-10-04T10:41:28Z

test/e2e/snaps/test-snap-confirm.spec.js

@@ -31,26 +34,9 @@ describe('Test Snap Confirm', function () {
        await driver.fill('#snapId1', 'npm:@metamask/test-snap-confirm');
        await driver.clickElement('#connectHello');

-        // switch to metamask extension and click connect
-        await driver.waitUntilXWindowHandles(2, 5000, 10000);


Connect to the snap test dapp in the fixture

PeterYinusa · 2022-10-04T10:41:38Z

test/e2e/snaps/test-snap-error.spec.js

@@ -32,25 +35,8 @@ describe('Test Snap Error', function () {
        await driver.fill('#snapId2', 'npm:@metamask/test-snap-error');
        await driver.clickElement('#connectError');

-        // switch to metamask extension and click connect


Connect to the snap test dapp in the fixture

PeterYinusa · 2022-10-04T10:41:45Z

test/e2e/snaps/test-snap-managestate.spec.js

@@ -40,25 +43,9 @@ describe('Test Snap manageState', function () {
        // connect the snap
        await driver.clickElement('#connectManageState');

-        // switch to metamask extension and click connect


Connect to the snap test dapp in the fixture

PeterYinusa · 2022-10-04T10:41:55Z

test/e2e/snaps/test-snap-notification.spec.js

@@ -40,25 +43,9 @@ describe('Test Snap Notification', function () {
        // connect the snap
        await driver.clickElement('#connectNotification');

-        // switch to metamask extension and click connect


Connect to the snap test dapp in the fixture

metamaskbot · 2022-10-04T11:36:58Z

Builds ready [be3c4d7]

builds: chrome, firefox, opera
builds (beta): chrome, firefox, opera
builds (flask): chrome, firefox, opera
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7, 8
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (2506 ± 145 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	102	186	124	23	11
		domContentLoaded	1910	3029	2485	303	146
		load	1910	3054	2506	302	145
		domInteractive	1910	3029	2485	303	146

test/e2e/fixture-builder.js

metamaskbot · 2022-10-04T13:59:26Z

Builds ready [3f7e590]

builds: chrome, firefox, opera
builds (beta): chrome, firefox, opera
builds (flask): chrome, firefox, opera
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7, 8
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (2357 ± 116 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	93	149	117	17	8
		domContentLoaded	1677	2724	2328	247	119
		load	1744	2747	2357	242	116
		domInteractive	1677	2724	2328	247	119

test/e2e/fixture-builder.js

test/e2e/tests/from-import-ui.spec.js

PeterYinusa · 2022-10-12T11:48:46Z

test/e2e/snaps/test-snap-update.spec.js

        let windowHandles = await driver.getAllWindowHandles();
        const extensionPage = windowHandles[0];
        await driver.switchToWindowWithTitle(
          'MetaMask Notification',
          windowHandles,
        );
-        await driver.clickElement(


Connect to the snap test dapp in the fixture

PeterYinusa · 2022-10-12T11:53:20Z

test/e2e/tests/failing-contract.spec.js

@@ -18,7 +19,19 @@ describe('Failing contract interaction ', function () {
    await withFixtures(
      {
        dapp: true,
-        fixtures: 'connected-state',
+        fixtures: new FixtureBuilder()
+          .withNetworkController({


The send failing transaction is a type 2 transaction

hide `Protect your funds` dialog remove browser environment remove default network details hide dismiss seed backup reminder recursively merges fixture data

wait for window

PeterYinusa · 2022-10-27T16:50:09Z

test/e2e/tests/chain-interactions.spec.js

-        const networkDisplay = await driver.findElement('.network-display');
+        const networkDisplay = await driver.findElement(
+          '[data-testid="network-display"]',
+        );


Nothings changed here, this is the same element as the original test, just using a different selector

Gudahtt

LGTM!

metamaskbot · 2022-10-27T17:48:29Z

Builds ready [2327778]

builds: chrome, firefox, opera
builds (beta): chrome, firefox, opera
builds (flask): chrome, firefox, opera
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5, 6, 7
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (2143 ± 99 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	96	2115	215	436	209
		domContentLoaded	1806	2641	2122	200	96
		load	1806	2641	2143	206	99
		domInteractive	1806	2641	2122	200	96

PeterYinusa commented Oct 3, 2022

View reviewed changes

PeterYinusa commented Oct 4, 2022

View reviewed changes

PeterYinusa marked this pull request as ready for review October 4, 2022 11:39

PeterYinusa requested a review from a team as a code owner October 4, 2022 11:39

PeterYinusa requested a review from NidhiKJha October 4, 2022 11:39

PeterYinusa added the needs-qa Label will automate into QA workspace label Oct 4, 2022

seaona reviewed Oct 4, 2022

View reviewed changes

test/e2e/fixture-builder.js Show resolved Hide resolved

Gudahtt reviewed Oct 4, 2022

View reviewed changes

test/e2e/fixture-builder.js Outdated Show resolved Hide resolved

test/e2e/tests/from-import-ui.spec.js Outdated Show resolved Hide resolved

test/e2e/tests/from-import-ui.spec.js Outdated Show resolved Hide resolved

PeterYinusa marked this pull request as draft October 7, 2022 10:12

PeterYinusa mentioned this pull request Oct 7, 2022

Increase e2e test coverage #13478

Closed

53 tasks

PeterYinusa force-pushed the e2e-test-fixtures branch from 3f7e590 to 3f502af Compare October 11, 2022 18:43

PeterYinusa commented Oct 12, 2022

View reviewed changes

PeterYinusa added 17 commits October 27, 2022 16:11

dismiss 3box whats new

0eca839

remove redundant code

360e487

move docs

165a4b0

remove unused code

3e87c44

token detection

5ffddaa

use default timeout

4fe7963

remove redundant code

2451231

Update fixture builder

9d84f8a

hide `Protect your funds` dialog remove browser environment remove default network details hide dismiss seed backup reminder recursively merges fixture data

add token to tokencontroller

3b5fda3

remove network details

2151515

add missing identities to preference controller

fb7de3a

remove duplicate properties

abeb5e8

update bip-32 to use fixturebuilder

5caaf10

alphabetise snap permissions

09004ff

update get snaps to use fixturebuilder

9e96ec2

Update test-snap-bip-32.spec.js

c77c8cc

wait for window

add popular network state

ce88224

PeterYinusa dismissed stale reviews from seaona, Gudahtt, and jpuri via ce88224 October 27, 2022 15:53

PeterYinusa force-pushed the e2e-test-fixtures branch from 95e4572 to ce88224 Compare October 27, 2022 15:53

PeterYinusa added 2 commits October 27, 2022 17:38

update test

5af2a41

lint

2327778

PeterYinusa commented Oct 27, 2022

View reviewed changes

Gudahtt approved these changes Oct 27, 2022

View reviewed changes

seaona approved these changes Oct 28, 2022

View reviewed changes

PeterYinusa merged commit 0b4532e into develop Oct 28, 2022

PeterYinusa deleted the e2e-test-fixtures branch October 28, 2022 08:42

github-actions bot locked and limited conversation to collaborators Oct 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

E2e test fixtures #16061

E2e test fixtures #16061

PeterYinusa commented Oct 3, 2022 •

edited

Loading

github-actions bot commented Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 3, 2022

PeterYinusa Oct 4, 2022

PeterYinusa Oct 4, 2022

PeterYinusa Oct 4, 2022

PeterYinusa Oct 4, 2022

PeterYinusa Oct 4, 2022

metamaskbot commented Oct 4, 2022

metamaskbot commented Oct 4, 2022

PeterYinusa Oct 12, 2022

PeterYinusa Oct 12, 2022

PeterYinusa Oct 27, 2022 •

edited

Loading

Gudahtt left a comment

metamaskbot commented Oct 27, 2022

E2e test fixtures #16061

E2e test fixtures #16061

Conversation

PeterYinusa commented Oct 3, 2022 • edited Loading

Explanation

More Information

Screenshots/Screencaps

Before

After

Manual Testing Steps

Pre-Merge Checklist

+ If there are functional changes:

github-actions bot commented Oct 3, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

metamaskbot commented Oct 4, 2022

metamaskbot commented Oct 4, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PeterYinusa Oct 27, 2022 • edited Loading

Choose a reason for hiding this comment

Gudahtt left a comment

Choose a reason for hiding this comment

metamaskbot commented Oct 27, 2022

PeterYinusa commented Oct 3, 2022 •

edited

Loading

PeterYinusa Oct 27, 2022 •

edited

Loading