Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance Tests Workflow: Use latest WP branch for release tests #32277

Merged
merged 1 commit into from
May 27, 2021

Conversation

ockham
Copy link
Contributor

@ockham ockham commented May 27, 2021

Description

Supersedes #32244.

As @youknowriad mentioned in Slack:

[R]elease perf jobs are breaking https://github.com/WordPress/gutenberg/runs/2676154517?check_suite_focus=true. If I have to guess, I think it's because the base WP install used comes from trunk and the fixes for the conflicts introduced so solve the conflicts are not included in both the previous release's branch and wp/5.7 branch. Ideal fix would be to rely on the 5.7 (previous wp release) as base branch instead of latest trunk like all the e2e tests jobs.

Turns out the performance script has a --tests-branch arg that is already used to set the branch from which we run the WP env for those tests:

run: ./bin/plugin/cli.js perf --ci $GITHUB_SHA trunk --tests-branch $GITHUB_SHA

It's used at

if ( !! options.testsBranch ) {
log(
'>> Fetching the ' +
formats.success( options.testsBranch ) +
' branch'
);
await git.checkoutRemoteBranch(
performanceTestDirectory,
options.testsBranch
);
}
(and below).

How has this been tested?

We'll only really know once we release the next GB version.

@ockham ockham added [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Performance Related to performance efforts labels May 27, 2021
@ockham ockham requested review from mcsf and youknowriad May 27, 2021 14:39
@ockham ockham self-assigned this May 27, 2021
Copy link
Contributor

@mcsf mcsf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's give it a go.

@ockham ockham marked this pull request as ready for review May 27, 2021 14:50
@ockham ockham merged commit 14a52d8 into trunk May 27, 2021
@ockham ockham deleted the fix/release-performance-tests branch May 27, 2021 15:20
@github-actions github-actions bot added this to the Gutenberg 10.8 milestone May 27, 2021
@@ -56,4 +56,4 @@ jobs:
WP_VERSION=$(grep -oP "$TESTED_UP_TO_REGEX" ./readme.txt)
IFS='.' read -r -a WP_VERSION_ARRAY <<< "$WP_VERSION"
WP_BRANCH="wp/${WP_VERSION_ARRAY[0]}.${WP_VERSION_ARRAY[1]}"
./bin/plugin/cli.js perf --ci $WP_BRANCH $PREVIOUS_RELEASE_BRANCH $CURRENT_RELEASE_BRANCH
./bin/plugin/cli.js perf --ci $WP_BRANCH $PREVIOUS_RELEASE_BRANCH $CURRENT_RELEASE_BRANCH --tests-branch $WP_BRANCH
Copy link
Contributor

@youknowriad youknowriad May 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I disagree with this one :) That's not what I suggested, I think we should always use "trunk" for the performance tests themselves, the issue is with the base branch used in the docker container which means we'll get a recent WP install (trunk) with all the new APIs with an old Gutenberg plugin wp/5.7 branch causing conflicts. I think @mcsf's PR solves that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Performance Related to performance efforts
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants