Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark test_stress_creation_and_deletion as xfail #5393

Merged

Conversation

jrbourbeau
Copy link
Member

xref #5388

cc @crusaderky

@crusaderky
Copy link
Collaborator

crusaderky commented Oct 5, 2021

I'm running stress testing to see if the flaky flag actually makes the test pass. We've had several cases in the past where a flaky test would either pass on the first go or fail 10 times in a row (which highlights that the whole VM, container, or process is borked).

My stress test will take 1-2 days to run. Happy to have this merged into main as a tentative fix while we wait.

@jrbourbeau
Copy link
Member Author

That makes sense. FWIW anecdotally this seems to help based on the CI runs for this commit that already finished over in my fork of distributed. Though I agree we should revisit based on the results on your stress tests (thank you for taking the time/effort to run those btw)

@jakirkham
Copy link
Member

jakirkham commented Oct 5, 2021

Yeah it seems it still fails in this CI job

Also seeing this failure ( #3574 ) in this CI job.

@crusaderky
Copy link
Collaborator

As I feared.

distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion RERUN [ 86%]
distributed/tests/test_stress.py::test_stress_creation_and_deletion FAILED [ 86%]

Copy link
Collaborator

@crusaderky crusaderky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marker should be changed to xfail

@jrbourbeau
Copy link
Member Author

Thanks for reviewing @jakirkham @crusaderky! CI failures are unrelated

@jrbourbeau jrbourbeau changed the title Mark test_stress_creation_and_deletion as flaky Mark test_stress_creation_and_deletion as xfail Oct 6, 2021
@jrbourbeau jrbourbeau merged commit a13321d into dask:main Oct 6, 2021
@jrbourbeau jrbourbeau deleted the test_stress_creation_and_deletion-flaky branch October 6, 2021 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants