make functional test cassettes work in any order

README.md

@@ -292,16 +292,41 @@ To individually run the unit tests, run `make test` to run the suite in parallel
 
 Functional tests are run alone using `make test-functional` and otherwise will be ran along with `make check`.
 
-Some of the tests appear to get into a state that reliably causes subsequent tests to crash. Such tests have been isolated and are clearly marked. The Makefile is used to ensure we exercise them in a completely new process.
-Use the `qtbot` object to drive the UI. This is part of the [pytest-qt](https://pytest-qt.readthedocs.io/en/latest/) package.
+We use `qtbot`, bundled with the [pytest-qt](https://pytest-qt.readthedocs.io/en/latest/) package, for UI interaction within our functional tests. We use [vcrpy](https://vcrpy.readthedocs.io/en/latest/) to replay the original response from the test server. These responses are stored in the `tests/functional/cassettes` directory. Our test TOTP is set to `994892` and stored in casettes.
 
-When writing tests that require the user to log in, on first run of the test
-you must make sure the TOTP value in `conftest.py` is correct for the time at which the test is run.
-For any further run of the test, this doesn't need to be the case since [vcrpy](https://vcrpy.readthedocs.io/en/latest/)
-will replay the original response from the test server. These responses are
-stored in the cassettes directory and should be committed to the git
-repository. Before committing, set the TOTP value in the cassette back to the value we use across all functional tests: `994892`.
+#### Generating new cassettes
 
+Some changes may require generating new cassettes, such as modifications to API endpoints. If you see the following warning, you may need to generate a new cassette or all new cassettes, depending on the change you made:
+
+```
+Can't overwrite existing cassette ('<path-to-cassette-for-a-functional-test>') in your current record mode ('once').
+```
+
+To generate new cassettes, follow these instructions:
+
+1. Before generating new cassettes that will be used to replay server interaction, bypass TOTP verification so that we can use the hard-coded value of `123456` in `tests/conftest.py`. You can do this by applying the following diff to the securedrop server, which you will be running in the next step:
+       ```diff
+       diff --git a/securedrop/models.py b/securedrop/models.py
+       index dcd26bbaf..50bc491b4 100644
+       --- a/securedrop/models.py
+       +++ b/securedrop/models.py
+       @@ -649,6 +649,7 @@ class Journalist(db.Model):
+
+                try:
+                    user = Journalist.query.filter_by(username=username).one()
+       +            return user
+                except NoResultFound:
+                    raise InvalidUsernameException(
+                        "invalid username '{}'".format(username))
+       (END)
+       ```
+2. Start the dev server: `NUM_SOURCES=0 make dev`
+3. Set up the dev server with data required for functional tests to pass and run in any order:
+    - Create 5 new sources, each with one submission that contains both a file and a message. The message should be set to `this is the message`. The file should be called `hello.txt` and contain a single line of text: `hello`.
+3. Delete all the old cassettes by running `rm -r tests/functional/cassettes` or just delete the cassettes you wish to regenerate.
+4. Run `make test-functional` which will regenerate cassettes for any that are missing in the `tests/functional/cassettes` folder.
+
+Note: After generating new cassettes, the test server will be in a state with only 3 sources. If for some reason you need to try again, make sure the server has the expected 5 sources, each with one submission that contains both the expected file and message before repeating the process of deleting cassettes and regenerating them.
 
 ## Making a Release
 

tests/conftest.py

@@ -34,7 +34,7 @@
 # Modify cassettes to use the following TOTP code. For developing new tests,
 # you can modify this so you don't need to keep editing cassettes during
 # development.
-TOTP = "994892"
+TOTP = "123456"
 
 # Time (in milliseconds) to wait for these GUI elements to render.
 TIME_APP_START = 1000

tests/functional/cassettes/test_login_as_journalist.yaml

@@ -1,7 +1,7 @@
 interactions:
 - request:
     body: '{"username": "journalist", "passphrase": "correct horse battery staple
-      profanity oil chewy", "one_time_code": "493941"}'
+      profanity oil chewy", "one_time_code": "123456"}'
     headers:
       Accept:
       - '*/*'
@@ -17,16 +17,17 @@ interactions:
     uri: http://localhost:8081/api/v1/token
   response:
     body:
-      string: "{\n  \"expiration\": \"2020-02-17T21:45:20.569829Z\", \n  \"journalist_first_name\":
-        null, \n  \"journalist_last_name\": null, \n  \"journalist_uuid\": \"1eff0bb5-289b-4105-b3c1-ec93d5704db6\",
-        \n  \"token\": \"eyJhbGciOiJIUzI1NiIsImlhdCI6MTU4MTk0NzEyMCwiZXhwIjoxNTgxOTc1OTIwfQ.eyJpZCI6MX0.4h6pPAy4bl58Yc5Gim1LVdhvTYELvMfIWoZPlBR2ZZI\"\n}\n"
+      string: "{\n  \"expiration\": \"2020-08-14T05:05:30.772170Z\", \n  \"journalist_first_name\"\
+        : \"\", \n  \"journalist_last_name\": \"\", \n  \"journalist_uuid\": \"a93d176e-676c-4f1c-b0d4-8f9ca8bd0989\"\
+        , \n  \"token\": \"eyJhbGciOiJIUzI1NiIsImlhdCI6MTU5NzM1MjczMCwiZXhwIjoxNTk3MzgxNTMwfQ.eyJpZCI6MX0.q_rpjWQWKqmAYnHanqG5cSDczmURfFjNmiAIcSDw0jc\"\
+        \n}\n"
     headers:
       Content-Length:
-      - '317'
+      - '313'
       Content-Type:
       - application/json
       Date:
-      - Mon, 17 Feb 2020 13:45:20 GMT
+      - Thu, 13 Aug 2020 21:05:30 GMT
       Server:
       - Werkzeug/0.16.0 Python/3.5.2
     status: