Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: skip or fix flaky tests #23167

Merged
merged 67 commits into from
Aug 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
5d14498
skip 23162
rachelruderman Aug 5, 2022
1a3ac28
skip 23101
rachelruderman Aug 5, 2022
9e1319d
skip 23099
rachelruderman Aug 5, 2022
111fc22
skip 23160, add comment
rachelruderman Aug 5, 2022
1bc704a
skip 23159
rachelruderman Aug 5, 2022
9e8da12
skip 23158
rachelruderman Aug 5, 2022
e704752
skip 23157
rachelruderman Aug 5, 2022
7e0a41d
skip 23156
rachelruderman Aug 5, 2022
85e5127
skip 23155
rachelruderman Aug 5, 2022
426ea3b
skip 23154
rachelruderman Aug 5, 2022
70921e9
skip 23153
rachelruderman Aug 5, 2022
db255e5
skip 23152
rachelruderman Aug 5, 2022
73770c5
skip 23151
rachelruderman Aug 5, 2022
445dcec
skip 23150
rachelruderman Aug 5, 2022
600b94c
skip 23149
rachelruderman Aug 5, 2022
3e1de0d
skip 23148
rachelruderman Aug 5, 2022
fa4fac9
Update net_stubbing.cy.ts
rachelruderman Aug 5, 2022
521c501
skip 23100
rachelruderman Aug 5, 2022
3e82dc8
skip 23201
rachelruderman Aug 8, 2022
c2d0f54
skip 23203
rachelruderman Aug 8, 2022
d05954c
Merge branch 'develop' into fix-or-skip-flaky-tests
rachelruderman Aug 9, 2022
92ce06b
skip 23220
rachelruderman Aug 9, 2022
bec6359
update vscode comments
rachelruderman Aug 9, 2022
dfc55c4
Merge branch 'fix-or-skip-flaky-tests' of github.com:cypress-io/cypre…
rachelruderman Aug 9, 2022
cfdae38
skip 23245,23247,23246,23214
rachelruderman Aug 10, 2022
ee77ef7
skip 23248
rachelruderman Aug 10, 2022
32fbdb6
skip 23250
rachelruderman Aug 10, 2022
b267d7c
add branch to circle.yml to allow testing on windows
rachelruderman Aug 10, 2022
f039462
skip 23252
rachelruderman Aug 10, 2022
04a1362
Merge branch 'develop' into fix-or-skip-flaky-tests
rachelruderman Aug 10, 2022
45f7c93
fix percy warnings and errors
rachelruderman Aug 10, 2022
17d9a94
Update yarn.lock
rachelruderman Aug 10, 2022
885d5bf
downgrade to percy/cli 1.2.0
rachelruderman Aug 10, 2022
1cbc0a7
skip 23264
rachelruderman Aug 10, 2022
0b6b4fa
fix addLog flake
rachelruderman Aug 11, 2022
21ba76f
Add branch to mainBuildFilters
rachelruderman Aug 11, 2022
dd4ad70
percy version bump
rachelruderman Aug 11, 2022
cf20d11
Merge branch 'develop' into fix-or-skip-flaky-tests
rachelruderman Aug 11, 2022
78ce539
use percy 1.2.0
rachelruderman Aug 11, 2022
88e9ace
Fix error output 23248
rachelruderman Aug 11, 2022
84b9e6d
fix 23145
rachelruderman Aug 11, 2022
5396601
hopeful fix 23264
rachelruderman Aug 11, 2022
c3dc991
fix 23101
rachelruderman Aug 11, 2022
3444562
fix 23150
rachelruderman Aug 11, 2022
587760a
Fix 23160
rachelruderman Aug 11, 2022
a4adee9
Fix some flaky tests from 23150
rachelruderman Aug 11, 2022
319b867
skip 23303
rachelruderman Aug 11, 2022
3fa50d0
Merge branch 'develop' into fix-or-skip-flaky-tests
rachelruderman Aug 12, 2022
2de35cf
Remove skip for 23248
rachelruderman Aug 12, 2022
74a7f16
use var instead of done
rachelruderman Aug 12, 2022
f281b51
Fix "can delay with deprecated delayMs param"
rachelruderman Aug 12, 2022
f9cc637
skip 23305
rachelruderman Aug 12, 2022
30ab403
skip 23306
rachelruderman Aug 12, 2022
c90fc9b
Add
rachelruderman Aug 12, 2022
2a81bd8
Revert "Add"
rachelruderman Aug 12, 2022
ae153a0
fix diffs
rachelruderman Aug 12, 2022
1eb489b
remove 10.0-release from branchesToAlwaysFinalize
rachelruderman Aug 12, 2022
2554779
Merge branch 'develop' into fix-or-skip-flaky-tests
rachelruderman Aug 12, 2022
663f9f7
fix 23318
rachelruderman Aug 12, 2022
375ae11
skip 23318
rachelruderman Aug 12, 2022
c5b33f3
revert try/catch
rachelruderman Aug 12, 2022
171d396
skip 23317
rachelruderman Aug 12, 2022
e14fb42
rerun workflow
rachelruderman Aug 12, 2022
1672431
Bump timeout for 'your tests are loading'. Possible fix for 23306
rachelruderman Aug 12, 2022
9fb50f3
rerun workflow
rachelruderman Aug 13, 2022
6294021
merge develop
rachelruderman Aug 15, 2022
e73d982
Merge branch 'develop' into fix-or-skip-flaky-tests
rachelruderman Aug 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .percy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ snapshot:
- 1280
min-height: 1024
discovery:
network-idle-timeout: 1000
network-idle-timeout: 750
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addresses Percy non-failing error:

Screen Shot 2022-08-15 at 10 23 05 AM

5 changes: 4 additions & 1 deletion circle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ mainBuildFilters: &mainBuildFilters
branches:
only:
- develop
- reapply-state-refactor
- fix-or-skip-flaky-tests

# usually we don't build Mac app - it takes a long time
# but sometimes we want to really confirm we are doing the right thing
Expand Down Expand Up @@ -63,7 +65,8 @@ windowsWorkflowFilters: &windows-workflow-filters
or:
- equal: [ develop, << pipeline.git.branch >> ]
- equal: [ linux-arm64, << pipeline.git.branch >> ]
- equal: [ 'tbiethman/22272-globbing-working-dir', << pipeline.git.branch >> ]
- equal: [ 'fix-or-skip-flaky-tests', << pipeline.git.branch >> ]
- equal: [ 'lmiller/windows-vite-fix', << pipeline.git.branch >> ]
- matches:
pattern: "-release$"
value: << pipeline.git.branch >>
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@
"@graphql-tools/wrap": "8.1.1",
"@octokit/auth-app": "3.6.1",
"@octokit/core": "3.6.0",
"@percy/cli": "1.1.0",
"@percy/cli": "1.2.0",
"@percy/cypress": "^3.1.1",
"@semantic-release/changelog": "5.0.1",
"@semantic-release/git": "9.0.0",
Expand Down
2 changes: 1 addition & 1 deletion packages/app/.percy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ snapshot:
- 1280
min-height: 1024
discovery:
network-idle-timeout: 1000
network-idle-timeout: 750
3 changes: 2 additions & 1 deletion packages/app/cypress/e2e/cypress-in-cypress-component.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,8 @@ describe('Cypress In Cypress CT', { viewportWidth: 1500, defaultCommandTimeout:
cy.get('[data-model-state="passed"]').should('contain', 'renders the test component')
})

it('redirects to the specs list with error if a spec is not found', () => {
// TODO: fix flaky test https://github.com/cypress-io/cypress/issues/23159
it.skip('redirects to the specs list with error if a spec is not found', () => {
cy.visitApp()
const { title, intro, explainer } = defaultMessages.specPage.noSpecError
const badFilePath = 'src/DoesNotExist.spec.js'
Expand Down
3 changes: 2 additions & 1 deletion packages/app/cypress/e2e/cypress-in-cypress-run-mode.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ describe('Cypress In Cypress - run mode', { viewportWidth: 1200 }, () => {
cy.percySnapshot()
})

it('component testing run mode spec runner header is correct', () => {
// TODO: fix flaky test https://github.com/cypress-io/cypress/issues/23306
it.skip('component testing run mode spec runner header is correct', () => {
cy.scaffoldProject('cypress-in-cypress')
cy.findBrowsers()
cy.openProject('cypress-in-cypress')
Expand Down
3 changes: 2 additions & 1 deletion packages/app/cypress/e2e/specs_list_e2e.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,8 @@ describe('App: Spec List (E2E)', () => {
cy.get('button').contains('23 Matches')
})

it('saves the filter when navigating to a spec and back', function () {
// TODO: fix flaky test https://github.com/cypress-io/cypress/issues/23305
it.skip('saves the filter when navigating to a spec and back', function () {
const targetSpecFile = 'accounts_list.spec.js'

clearSearchAndType(targetSpecFile)
Expand Down
2 changes: 1 addition & 1 deletion packages/app/cypress/e2e/support/execute-spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ export const waitForSpecToFinish = (expectedResults) => {
cy.get('.failed > .num').should('contain', '--')

// Then ensure the tests are running
cy.contains('Your tests are loading...').should('not.exist')
cy.contains('Your tests are loading...', { timeout: 10000 }).should('not.exist')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopeful fix for #23306


// Then ensure the tests have finished
cy.get('[aria-label="Rerun all tests"]', { timeout: 30000 })
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ describe('GitDataSource', () => {
sinon.restore()
})

it(`gets correct status for files on ${os.platform()}`, async () => {
// TODO: fix flaky test https://github.com/cypress-io/cypress/issues/23317
it.skip(`gets correct status for files on ${os.platform()}`, async () => {
const onBranchChange = sinon.stub()
const onGitInfoChange = sinon.stub()
const onError = sinon.stub()
Expand Down
6 changes: 4 additions & 2 deletions packages/driver/cypress/e2e/commands/actions/select.cy.js
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,8 @@ describe('src/cy/commands/actions/select', () => {
cy.get('#select-maps').select('de_train')
})

it('can forcibly click even when being covered by another element', (done) => {
// TODO: fix flaky test https://github.com/cypress-io/cypress/issues/23156
it.skip('can forcibly click even when being covered by another element', (done) => {
const select = $('<select><option>foo</option></select>').attr('id', 'select-covered-in-span').prependTo(cy.$$('body'))

$('<span>span on select</span>').css({ position: 'absolute', left: select.offset().left, top: select.offset().top, padding: 5, display: 'inline-block', backgroundColor: 'yellow' }).prependTo(cy.$$('body'))
Expand All @@ -173,7 +174,8 @@ describe('src/cy/commands/actions/select', () => {
cy.get('#select-covered-in-span').select('foobar', { timeout: 1000, interval: 60 })
})

it('can forcibly click even when element is invisible', (done) => {
// TODO: fix flaky test https://github.com/cypress-io/cypress/issues/23157
it.skip('can forcibly click even when element is invisible', (done) => {
const select = cy.$$('#select-maps').hide()

select.click(() => {
Expand Down
Loading