Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: reduce impact of flaky HTTP server tests #42926

Merged
merged 1 commit into from
May 2, 2022
Merged

test: reduce impact of flaky HTTP server tests #42926

merged 1 commit into from
May 2, 2022

Conversation

tniessen
Copy link
Member

Mark the tests as flaky on macOS and increase the timeout for other platforms (namely, FreeBSD).

Context: #42741 (comment)

Median success rate (refer to patch and separate tables above):

osx11 freebsd12-x64
master (2000ms) 91.0 % 83.8 %
patched (5000ms) 71.3 % 100 %

Refs: #42741

@tniessen tniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Apr 30, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
@tniessen
Copy link
Member Author

tniessen commented Apr 30, 2022 •
edited
Loading

Is PASS,FLAKY not the right configuration? node-test-commit-osx still fails with this. Should it be SKIP?

Removed the file extensions, maybe that works.

@tniessen tniessen force-pushed the flaky-test-http-server-x-timeout-keepalive branch from 35014d7 to 06c1abe Compare April 30, 2022 14:06
@targos
Copy link
Member

targos commented Apr 30, 2022

Removed the file extensions, maybe that works.

Yep:

node/test/parallel/parallel.status

Lines 3 to 5 in 02e0c17

# To mark a test as flaky, list the test name in the appropriate section
# below, without ".js", followed by ": PASS,FLAKY". Example:
# sample-test : PASS,FLAKY

@tniessen tniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 30, 2022
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/43796/

aduh95
aduh95 approved these changes Apr 30, 2022
View reviewed changes
Copy link
Contributor

@aduh95 aduh95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flakiness is also annoying on the GHA runner, and unfortunately this PR won't help with that I'm afraid. Oh well, at least it's improving things for the Jenkins CI, so LGTM.

@github-actions github-actions bot mentioned this pull request May 1, 2022
Open
29 tasks
@tniessen
Copy link
Member Author

tniessen commented May 1, 2022

The flakiness is also annoying on the GHA runner, and unfortunately this PR won't help with that I'm afraid.

@aduh95 I am not sure about that. #42741 (comment) suggested that GitHub Actions might respect the list of flaky tests.

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/43804/

@tniessen
Copy link
Member Author

tniessen commented May 1, 2022

CI is failing because of nodejs/build#2917.

@tniessen tniessen added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 1, 2022
@github-actions github-actions bot mentioned this pull request May 2, 2022
Open
30 tasks
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/43810/

@tniessen tniessen added the commit-queue Add this label to land a pull request using GitHub Actions. label May 2, 2022
@tniessen
Copy link
Member Author

tniessen commented May 2, 2022

CI is green but GitHub seems to be thinking that Jenkins is still running, and the commit queue refuses to run...

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/43826/

@aduh95 aduh95 removed the commit-queue Add this label to land a pull request using GitHub Actions. label May 2, 2022
@aduh95 aduh95 merged commit 0fc455a into nodejs:master May 2, 2022
@aduh95
Copy link
Contributor

aduh95 commented May 2, 2022

Landed in 0fc455a

@tniessen
Copy link
Member Author

tniessen commented May 2, 2022

Thank you @aduh95 :)

This was referenced May 3, 2022
Open
Open
Open
RafaelGSS pushed a commit that referenced this pull request May 10, 2022
@tniessen @RafaelGSS
test: reduce impact of flaky HTTP server tests
72b90fd 
Refs: #42741

PR-URL: #42926
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
@RafaelGSS RafaelGSS mentioned this pull request May 10, 2022
Merged
@daeyeon daeyeon mentioned this pull request May 23, 2022
Merged
@danielleadams
Copy link
Contributor

This PR is dependent on #42846, so marking as blocked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aduh95 aduh95 aduh95 approved these changes

@targos targos targos approved these changes

@lpinca lpinca lpinca approved these changes

@richardlau richardlau Awaiting requested review from richardlau

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@tniessen @nodejs-github-bot @targos @aduh95 @danielleadams @lpinca