Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pytest-flaky to retry fragile functional tests. #956

Closed
ntoll opened this issue Mar 19, 2020 · 2 comments · Fixed by #1001
Closed

Use pytest-flaky to retry fragile functional tests. #956

ntoll opened this issue Mar 19, 2020 · 2 comments · Fixed by #1001

Comments

@ntoll
Copy link
Contributor

ntoll commented Mar 19, 2020

Sometimes, on CI, our functional tests fail with errors like this:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file

Such random, impossible to reproduce and frustrating errors are caused by a problematic state in the CI server, rather than with our tests. For instance, the error quoted above is due to disk read/write problems on the server.

I suggest we use pytest-flaky as a way to retry such flaky tests.

@kushaldas
Copy link
Contributor

I saw the same error today:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)

@kushaldas
Copy link
Contributor

Now I am seeing this error in all random tests in both master and while working on https://github.com/freedomofpress/securedrop-client/tree/login_via_clicking_icon_functional_test I think I should just add flaky to all tests under functional

kushaldas added a commit that referenced this issue Mar 25, 2020
Now we are using pytest-flaky for all the functional tests.
redshiftzero pushed a commit that referenced this issue Mar 25, 2020
Now we are using pytest-flaky for all the functional tests.
redshiftzero added a commit that referenced this issue Mar 25, 2020
Fixes #956 Adds flaky for functinoal tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants