Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch performance tests to Playwright #52022

Merged
merged 46 commits into from
Aug 16, 2023
Merged

Switch performance tests to Playwright #52022

Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
a88918c
Run with Playwright in CI
WunderBart Jun 28, 2023
4ce6571
Print stdout on error
WunderBart Jun 28, 2023
f6d9851
Run build:packages to use e2e utils
WunderBart Jun 28, 2023
6fde09d
Extend timeout for the canvas spinner
WunderBart Jun 28, 2023
6f28c60
Add missing front end perf report
WunderBart Jun 28, 2023
b9b0440
Remove obsolete step
WunderBart Jun 29, 2023
ea6891f
Make the typing testing consistent
WunderBart Jun 29, 2023
aeae2ec
Improve post editor perf testing consistency
WunderBart Jul 4, 2023
309b9a2
Do not report slow tests
WunderBart Jul 5, 2023
b47d2b7
Use the same methodology for measuring post and site editors metrics
WunderBart Jul 5, 2023
0427a37
Make the results table transformer more readable
WunderBart Jul 5, 2023
542456e
Streamline the results processing p.1
WunderBart Jul 5, 2023
c06ab19
Improve visual step separation a tad :)
WunderBart Jul 5, 2023
df1216e
First step to separate metrics from the runner
WunderBart Jul 6, 2023
2e13186
Define the results file suffix
WunderBart Jul 6, 2023
ed1e0cc
Merge remote-tracking branch 'origin' into refactor/playwright-perfor…
WunderBart Jul 6, 2023
fecc1a0
cosmetic
WunderBart Jul 6, 2023
4a394d8
Fix matching pattern typo
WunderBart Jul 6, 2023
647ea57
Make performance.js comparison runner results-agnostic
WunderBart Jul 10, 2023
1a18006
Fix results handling
WunderBart Jul 10, 2023
f03e671
Use Playwright's attachments API to handle results
WunderBart Jul 10, 2023
5db75c3
Upload raw results as well
WunderBart Jul 10, 2023
da43808
Be consistent with calc utils
WunderBart Jul 11, 2023
00f641d
Streamline results handling
WunderBart Jul 11, 2023
2b9a5ac
Use the same sampling convention for front-end metrics
WunderBart Jul 11, 2023
1f8ecd1
Fix the metrics output in perf runner
WunderBart Jul 11, 2023
d649aac
tmp: compare performance with base branch
WunderBart Jul 12, 2023
ae40542
tmp: compare performance with base branch (run 2)
WunderBart Jul 12, 2023
18fe132
tmp: compare performance with base branch (run 3)
WunderBart Jul 12, 2023
8f15066
Revert "tmp: compare performance with base branch"
WunderBart Jul 12, 2023
b1d7f83
Remove unused code and do the switch
WunderBart Jul 12, 2023
e37cbc2
Merge remote-tracking branch 'origin' into refactor/playwright-perfor…
WunderBart Jul 12, 2023
2c305da
Fix site editor typing spec
WunderBart Jul 26, 2023
c9c625e
Merge remote-tracking branch 'origin' into refactor/playwright-perfor…
WunderBart Jul 26, 2023
bbae249
Add inline comment
WunderBart Jul 27, 2023
894896b
Expain why we need larger timeout for site editor visit util
WunderBart Jul 27, 2023
7ac254e
Use dirent for getting files from dir
WunderBart Jul 27, 2023
84bb262
Do not use the browser.newPage shortcut
WunderBart Jul 27, 2023
182fbc1
Merge remote-tracking branch 'origin' into refactor/playwright-perfor…
WunderBart Jul 27, 2023
1cf298a
Use trace buffer instead of tmp file
WunderBart Jul 27, 2023
f787f1e
Merge remote-tracking branch 'origin' into refactor/playwright-perfor…
WunderBart Aug 14, 2023
5a68bbc
Fix 'unterminated string' error
WunderBart Aug 14, 2023
d0d81fb
Fix 'unterminated string' error p.2
WunderBart Aug 14, 2023
37ef946
Make sure sanitized branch name is used for reading results json
WunderBart Aug 16, 2023
127b21e
Merge remote-tracking branch 'origin' into refactor/playwright-perfor…
WunderBart Aug 16, 2023
568d9d7
Prevent logging in via test.use() instead of creating new context
WunderBart Aug 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/performance.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:
uses: ./.github/setup-node

- name: Compare performance with trunk
if: github.event_name == 'pull_request'
if: ${{ false }}
run: ./bin/plugin/cli.js perf $GITHUB_SHA trunk --tests-branch $GITHUB_SHA

- name: Compare performance with current WordPress Core and previous Gutenberg versions
Expand All @@ -57,7 +57,7 @@ jobs:
./bin/plugin/cli.js perf "wp/$WP_MAJOR" "$PREVIOUS_RELEASE_BRANCH" "$CURRENT_RELEASE_BRANCH" --tests-branch $GITHUB_SHA --wp-version "$WP_MAJOR"

- name: Compare performance with base branch
if: github.event_name == 'push'
if: github.event_name == 'pull_request'
# The base hash used here need to be a commit that is compatible with the current WP version
# The current one is 34af5829ac9edb31833167ff6a3b51bea982999c and it needs to be updated every WP major release.
# It is used as a base comparison point to avoid fluctuation in the performance metrics.
Expand All @@ -80,7 +80,7 @@ jobs:
uses: actions/upload-artifact@83fd05a356d7e2593de66fc9913b3002723633cb # v3.1.1
with:
name: performance-results
path: ${{ env.WP_ARTIFACTS_PATH }}/*.performance-results.json
path: ${{ env.WP_ARTIFACTS_PATH }}/*.performance-results*.json

- name: Publish performance results
if: github.event_name == 'push'
Expand Down
Loading