E2E Tests: retry flaky tests #7758
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Size Change: 0 B Total Size: 2.11 MB ℹ️ View Unchanged
|
spacedmonkey
added
P2
Codecov Report
@@ Coverage Diff @@
## main #7758 +/- ##
==========================================
+ Coverage 82.56% 84.37% +1.80%
==========================================
Files 1155 1158 +3
Lines 17428 17449 +21
==========================================
+ Hits 14390 14722 +332
+ Misses 3038 2727 -311
Flags with carried forward coverage won't be shown. Click here to find out more.
|
swissspidy
approved these changes
May 27, 2021
kevin940726
reviewed
May 30, 2021
| @@ -112,6 +112,11 @@ const pageEvents = []; | |||
| // The Jest timeout is increased because these tests are a bit slow | |||
| jest.setTimeout(PUPPETEER_TIMEOUT || 100000); | |||
|
|
|||
| // Retry flaky tests at most 2 times in CI (off by 1). | |||
There was a problem hiding this comment.
I don't think this is correct though. I believe retryTimes(3) means it will at most run the test 4 times, 1 normal run, then 3 retries.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context
Summary
Prior art: WordPress/gutenberg#31682
Requires jest-circus test runner, which is the default since Jest 27.
Retry 3 means that test is run 3 times, normally and 2 other times.
Relevant Technical Choices
To-do
User-facing changes
Testing Instructions
QA
This PR can be tested by following these steps:
UAT
This PR can be tested by following these steps:
Reviews
Does this PR have a security-related impact?
Does this PR change what data or activity we track or use?
Does this PR have a legal-related impact?
Checklist
Type: XYZlabel to the PRFixes #7549