make functional test cassettes work in any order

README.md

@@ -286,22 +286,64 @@ To run everything, run:
 make check
 ```
 
-To individually run the unit tests, run `make test` to run the suite in parallel (fast), or run `make test-random` to run the tests in random order (slower, but this is what `make check` runs and what runs in CI).
+To individually run the unit tests, run `make test` to run the suite in parallel (fast), or run
+`make test-random` to run the tests in random order (slower, but this is what `make check` runs and
+what runs in CI).
 
 ### Functional Tests
 
-Functional tests are run alone using `make test-functional` and otherwise will be ran along with `make check`.
+Functional tests are run alone using `make test-functional` and otherwise will be ran along with
+`make check`.
 
-Some of the tests appear to get into a state that reliably causes subsequent tests to crash. Such tests have been isolated and are clearly marked. The Makefile is used to ensure we exercise them in a completely new process.
-Use the `qtbot` object to drive the UI. This is part of the [pytest-qt](https://pytest-qt.readthedocs.io/en/latest/) package.
+We use `qtbot`, bundled with the [pytest-qt](https://pytest-qt.readthedocs.io/en/latest/) package,
+for UI interaction within our functional tests. We use
+[vcrpy](https://vcrpy.readthedocs.io/en/latest/) to replay the original response from the test
+server. These responses are stored in the `tests/functional/cassettes` directory. Our test TOTP is
+set to `994892` and stored in casettes.
 
-When writing tests that require the user to log in, on first run of the test
-you must make sure the TOTP value in `conftest.py` is correct for the time at which the test is run.
-For any further run of the test, this doesn't need to be the case since [vcrpy](https://vcrpy.readthedocs.io/en/latest/)
-will replay the original response from the test server. These responses are
-stored in the cassettes directory and should be committed to the git
-repository. Before committing, set the TOTP value in the cassette back to the value we use across all functional tests: `994892`.
+#### Generating new cassettes
 
+Some changes may require generating new cassettes, such as modifications to API endpoints. If you
+see the following warning, you may need to generate a new cassette or all new cassettes, depending
+on the change you made:
+
+```
+Can't overwrite existing cassette ('<path-to-cassette-for-a-functional-test>') in your
+current record mode ('once').
+```
+
+To generate new cassettes, follow these instructions:
+
+0. Before generating new cassettes that will be used to replay server interaction, bypass TOTP
+   verification so that we can use the hard-coded value of `994892` in `tests/conftest.py` by
+   applying this diff to the securedrop server that you will be running in step 1.
+       ```diff
+       diff --git a/securedrop/models.py b/securedrop/models.py
+       index dcd26bbaf..50bc491b4 100644
+       --- a/securedrop/models.py
+       +++ b/securedrop/models.py
+       @@ -649,6 +649,7 @@ class Journalist(db.Model):
+
+                try:
+                    user = Journalist.query.filter_by(username=username).one()
+       +            return user
+                except NoResultFound:
+                    raise InvalidUsernameException(
+                        "invalid username '{}'".format(username))
+       (END)
+       ```
+1. Start the dev server `NUM_SOURCES=0 make dev`
+2. Set up the dev server with the data the functional tests need in order for all tests to run in
+   any order:
+    - Create 5 new sources, each with one submission that contains both a file and message
+    - The message should be set to `this is the message`
+    - The file should be called `hello.txt` and contain a single line of text: `hello`
+3. Delete the old cassettes: `rm -r tests/functional/cassettes/test_download_file.yaml`
+4. Run the functional tests to generate new cassettes: `make test-functional`
+
+Note: After generating new cassettes, the test server will be in a state with only 2 sources. If you
+need to run through the steps listed above again, make sure to re-add 3 new sources, each with one
+submission that contains both the expected file and message.
 
 ## Making a Release
 

tests/functional/cassettes/test_login_as_journalist.yaml

@@ -1,7 +1,7 @@
 interactions:
 - request:
     body: '{"username": "journalist", "passphrase": "correct horse battery staple
-      profanity oil chewy", "one_time_code": "493941"}'
+      profanity oil chewy", "one_time_code": "994892"}'
     headers:
       Accept:
       - '*/*'

tests/functional/cassettes/test_logout_as_journalist.yaml

@@ -17,16 +17,17 @@ interactions:
     uri: http://localhost:8081/api/v1/token
   response:
     body:
-      string: "{\n  \"expiration\": \"2020-02-20T22:11:20.428580Z\", \n  \"journalist_first_name\":
-        null, \n  \"journalist_last_name\": null, \n  \"journalist_uuid\": \"29c7756e-c0f9-4316-b42f-a0af9e20adb0\",
-        \n  \"token\": \"eyJpYXQiOjE1ODIyMDc4ODAsImV4cCI6MTU4MjIzNjY4MCwiYWxnIjoiSFMyNTYifQ.eyJpZCI6MX0.GX36N5A-cyf6bdvalil5_gRa9MnDgIZU_z4NgV8Vzcc\"\n}\n"
+      string: "{\n  \"expiration\": \"2020-08-12T13:46:20.554271Z\", \n  \"journalist_first_name\"\
+        : \"\", \n  \"journalist_last_name\": \"\", \n  \"journalist_uuid\": \"e169398c-ca8d-4fc5-9b0e-c42e15b21d66\"\
+        , \n  \"token\": \"eyJpYXQiOjE1OTcyMTExODAsImV4cCI6MTU5NzIzOTk4MCwiYWxnIjoiSFMyNTYifQ.eyJpZCI6MX0._cOdrvMQJNLm7gflWhQaDGP2W8ghXmazLPSukC-r3CE\"\
+        \n}\n"
     headers:
       Content-Length:
-      - '317'
+      - '313'
       Content-Type:
       - application/json
       Date:
-      - Thu, 20 Feb 2020 14:11:20 GMT
+      - Wed, 12 Aug 2020 05:46:20 GMT
       Server:
       - Werkzeug/0.16.0 Python/3.5.2
     status: